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EXECUTIVE  SUMMARY 


BACKGROUND 

The  U.S.  Coast  Guard  provides  for 
partial  credit  toward  certain  mariner 
licenses,  endorsements  or  renewals 
based  on  completing  "approved"  simula¬ 
tor  training.  Previous  studies  in 
this  series  have  produced  the  guide¬ 
lines  for  approval  of  Simulator  train¬ 
ing  for  cadets  and  for  senior  mariners 
who  are  ship's  company.  No  such 
guidelines  existed  for  approving  simu¬ 
lator  training  for  local  pilots.  In 
fact,  there  was  substantial  doubt 
among  pilots  as  to  whether  current 
simulator  technology  could  adequately 
display  the  shiphandling  precision 
needed  for  sound  pilot  training. 

THE  STUDY 

Dramatic  recent  advances  in  ship  simu¬ 
lator  capabilities,  along  with  con¬ 
firmed  success  of  pilot  simulator 
training  programs  in  Europe,  led  to 
this  research  project.  Its  objectives 
were:  First,  to  establish  whether 
valid  and  and  credible  pilot  simulator 
training  was  feasible  and,  second,  if 
it  was  credible,  to  formulate  approval 
guidelines  for  training  courses  to 
justify  licensing  credit.  The  student 
was  jointed  sponsored  and  funded  by 
the  U.S.  Coast  Guard  and  Maritime 
Administration.  The  Maritime  Adminis¬ 
tration's  Computer-Aided  Operations 
Research  Facility  (CAORF)  was  the 
research  site.  An  experimental  train¬ 
ing  program  was  developed  specifically 
to  measure  training  benefits.  (It  was 
not  designed  to  optimize  training, 
although  systematic  instructional  sys¬ 
tem  development  principles  were 


used.)  Two  senior  pilots  helped 
design  and  conduct  the  training  pro¬ 
gram.  The  twelve  trainees  were  pilots 
of  varying  experience  from  the  U.S. 
East,  Gulf,  and  West  coasts.  Training 
in  three  skill  areas:  Advanced  In¬ 
strumentation,  Emergency  Shiphandling 
and  Decisionmaking  (in  collision  emer¬ 
gencies)  was  administered.  Objective 
measures  of  shiphandling  performance 
were  taken  before  and  after  each  type 
of  training.  Pilot  opinions  were 
recorded  using  debriefing  question¬ 
naires.  Both  objective  data  and  sub¬ 
jective  opinions  were  analyzed  to 
reach  the  conclusions  of  this  study. 

CONCLUSIONS 

1.  Some  forms  of  simulator  training 
are  highly  effective  in  improving 
pilot  skills. 

2.  Experienced  pilots,  apprentices, 
and  qualified  pilots  with  limited 
experience  can  all  benefit  from 
simulator  training. 

3.  Different  types  of  training  are 
needed  for  pilots  with  different 
levels  of  experience;  however, 
pilots  can  benefit  by  mixing  dif¬ 
ferent  experience  levels  withtn  the 
same  class. 

4.  The  simulated  port  area  in  a  train¬ 
ing  program  does  not  have  to  be  a 
specific  real  port,  but  it  should 
present  challenges  that  are  rele¬ 
vant  to  each  trainee's  own  port. 

5.  Pilots  require  higher  fidelity 
training  simulation,  especially 


regarding  the  fine  points  of  ship 
motions,  than  do  ship's  company 
mariners. 

6.  The  experimental  training  program 
in  this  study  featured: 

a.  Small  class  size  (4-5  pilots). 

b.  A  mixture  of  senior  and  less 
experienced  pilots  in  the  same 
class. 

c.  A  seminar  format  with  the  in¬ 
structor  acting  as  a  seminar 
leader,  rather  than  as  a  author¬ 
itarian  teacher  lecturing  stu¬ 
dents. 

d.  Coordinators  who  were  highly 
experienced,  active  pilots  and 
who  were  also  familiar  with  the 
functioning  of  the  specific 
simulators. 

e.  Trainees  who  were  volunteers 
(not  forced  by  Government  to 
take  the  training). 

7.  Within  our  sample  of  twelve  pilots, 
the  great  majority  felt: 

a.  That  simulator  training  could  be 
beneficial  and  effective,  if 
properly  designed  and  conducted. 

b.  That  high  simulator  fidelity  is 
essential . 

c.  That  each  of  the  training  pro¬ 
gram  features  in  6.  above  is 
important  and  beneficial. 

d.  That,  during  the  experiment,  the 
amount  of  time  spent  on  each 
aspect  of  training  was  too  short 
and  allowed  too  few  practice 
runs. 

8.  The  training  in  this  study  was 
designed  to  measure  whether  any 


such  training  was  of  value.  A  pro¬ 
gram  to  optimize  training,  rather 
than  measurement,  would  offer  more 
problems  and  more  practice  on  each 
problem  than  this  experimental  pro¬ 
gram. 


9.  Identifying  specific  types  of 
training  for  pilots  (such  as  Ad¬ 
vanced  Instrumentation  and  Emer¬ 
gency  Shiphandling)  is  aided  by  a 
structured  research  approach  de¬ 
scribed  in  the  body  of  the  report. 


RECOMMENDATIONS  FOR  SIMULATOR  TRAIN¬ 
ING  POLICY  AND  FOR  FURTHER  RESEARCH 

This  study  did  not  require  policy  and 
future  research  recommendations;  how¬ 
ever,  the  following  are  useful  indica¬ 
tors  of  the  interests  of  the  study 
team  and  the  pilot  trainees. 

1.  The  Government  should  encourage, 
but  not  require,  simulator  training 
for  pilots. 

2.  Several  pilot  groups  for  whom 
training  would  be  particularly 
cost-effective  should  be  identi¬ 
fied.  Demonstration  training  pro¬ 
grams  should  be  cost-shared  by  the 
pilots  and  the  Government. 

3.  Additional  research  should  be  con¬ 
ducted,  specifically: 

a.  More  complete  identification  of 
the  types  of  simulator  training 
beneficial  for  pilots. 

b.  Development  of  a  high  quality 
generic  port  data  base  for 
training  a  wide  range  of  pilot 
skills. 

c.  Refinement  and  validation  of 
ship  motion  models  suitable  for 
simulator  training  for  pilots. 


RELATED  INFORMATION 


d.  At-sea  validation  of  the  bene¬ 
fits  of  simulator  training. 

4.  The  U.S.  Coast  Guard  is  not  plan¬ 
ning  to  pursue  any  such  lines  of 
research. 


The  International  Marine  Simulator 
Forum  (IMSF)  a  worldwide  assembly  of 
visual  ship  simulator  facilities,  is 
undertaking  an  international  survey  of 
pilot  simulator  training  needs  and 
capabilities. 


CHAPTER  1 


INTRODUCTION 


1.1  BACKGROUND 

The  ship  bridge/shiphandling  simulator 
has  become  an  important  tool  for  use 
in  research  and  training  within  the 
maritime  industry.  It  has  been  used 
successfully  to  train  all  levels  of 
deck  officers,  from  cadets  through 
masters.  In  some  cases,  simulator- 
based  training  is  a  cost-effective 
alternative  for  achieving  specific 
training  objectives,  while  in  other 
cases  it  enables  training  and  practice 
in  situations  that  cannot  be  adequate¬ 
ly  addressed  via  other  means  (e.g., 
due  to  safety  considerations,  or  lack 
of  training  control).  It  is  generally 
viewed  as  a  means  of  augmenting  exist¬ 
ing  training  programs.  A  variety  of 
shipping  companies,  for  example,  have 
made  available  simulator-based  train¬ 
ing  programs  for  their  senior  level 
deck  officers,  renting  time  at  the 
commercial  schools  world-wide.  At  the 
other  extreme,  all  deck  cadets  at  the 
U.S.  Merchant  Marine  Academy  now  par¬ 
ticipate  in  a  shiphandling  simulator- 
based  training  program,  using  the 
government-owned  Computer  Aided  Opera¬ 
tions  Research  Facility  (CAORF).  Sev¬ 
eral  deck  officer  labor  organizations, 
(e.g..  Marine  Engineers  Beneficial 
Association  -  District  Two;  Interna¬ 
tional  Organization  of  Masters,  Mates, 
and  Pilots)  have  developed  their  own 
shiphandling  simulator-based  training 
system  to  accommodate  members.  Hence, 
a  variety  of  shiphandling  training 
system  exist  world-wide  to  accommodate 
a  growing  demand. 


Pilots,  who  as  a  group  represent  the 
high  extreme  of  shiphandling  skills, 
have  availed  themselves  of  simulator- 
based  training  only  to  a  minor  ex¬ 
tent.  This  is  due  to  several  reasons, 
such  as  the  lack  of  experience  working 
with  simulators,  the  level  of  simula¬ 
tion  technology  available,  and  the 
lack  of  definitive  evidence  demon¬ 
strating  the  effectiveness  of  simula¬ 
tor-based  training  for  pilots.  The 
most  notable  uses  of  simulator-based 
training  for  pilots  occur  in  Europe  — 
the  Rotterdam  Europort  pilots  who 
train  in  Wageningen,  Netherlands;  the 
simulation  facility  at  the  Hochschule 
Fur  Nautik  in  Bremen,  Germany  which 
has  trained  German  pilots;  and  the 
Port  Revel  model  ship  basin  in 
Grenoble,  France  which  has  been  used 
to  train  various  groups  of  pilots, 
including  pilots  from  the  United 
States.  The  Europort  pilots  have  been 
training  on  the  simulator  at  Wagenin¬ 
gen  for  about  10  years.  After  ini¬ 
tially  trying  the  simulator  with  some 
degree  of  reservation,  the  Dutch  have 
come  to  rely  on  simulator-based  train¬ 
ing  as  an  important  part  of  pilot  pre¬ 
paration  and  refresher  training.  Each 
Europort  pilot,  who  is  a  highly  expe¬ 
rienced  professional  bringing  large 
tankers  in  and  out  of  Europort,  under¬ 
goes  refresher  training  on  the  simula¬ 
tor  every  two  years.  Their  training 
focuses  on  the  use  of  electronic  aids 
to  navigation  and  difficult  large  ship 
maneuvers  in  a  narrow  channel  (e.g., 
turning  a  VLCC  around).  Although 
there  are  several  notable  uses  of 
simulator-based  training  by  pilots, 
the  effectiveness,  or  lack  of 


effectiveness,  of  this  training  media 
for  United  States  pilots  has  not  been 
demonstrated. 

Several  major  issues  have  affected  the 
use  of  simulator-based  training  by 
U.S.  pilots.  The  most  dominant  of 
these  is  the  lack  of  detailed  objec¬ 
tive  information  regarding  the  poten¬ 
tial  use  of  this  training  media  by 
pilots,  its  capabilities  and  limita¬ 
tions,  requisite  simulation  character¬ 
istics  and  quality,  and  so  on  —  that 
is,  details  concerning  its  training 
effectiveness  for  pilots.  Based  on 
previous  research  by  the  U.S.  Coast 
Guard  and  U.S.  Maritime  Administra¬ 
tion,  it  was  expected  that  simulator- 
based  training  would  be  effective  for 
addressing  certain  piloting  skills 
such  as  emergency  shiphandling. 
Furthermore,  it  was  expected  that  the 
effectiveness  of  simulator-based 
training  of  certain  piloting  skills 
would  depend  on  particular  pilots 
backgrounds  (e.g.,  the  amount  of  expe¬ 
rience).  The  greatest  potential  value 
of  the  simulator-based  training  of 
pilots  was  not  as  a  replacement  for 
parts  of  current  pilot  training  pro¬ 
grams,  since  these  are  believed  to  be 
effective.  Rather,  the  potential  was 
to  augment  current  training  programs 
where  desired  by  particular  groups  of 
pilots.  Hence,  the  necessity  existed 
to  develop  objective  information 
detailing  the  potential  effective  use 
of  simulator-based  training  by  pilots, 
and  to  provide  guidance  information  to 
assist  pilots  in  using  this  technology. 

The  U.S.  Coast  Guard  and  U.S.  Maritime 
Administration,  in  response  to  the 
growing  availability  of  the  ship- 
handling  simulator  training  techno¬ 
logy,  Initiated  research  to  determine 
the  cost-effectiveness  of  such  train¬ 
ing  media.  This  research,  which  has 
been  conducted  by  the  Computer  Aided 
Operations  Research  Facility  (CAORF), 
has  included  experiments  investigating 


the  effectiveness  of  simulator-based 
training  of  maritime  academy  cadets, 
chief  mates,  and  masters.  A  major 
product  of  this  research  is  a  document 
which  contains  shiphandling  simulator 
training  system  design  and  application 
guidelines  developed  for  use  by  senior 
deck  officers.  The  guidelines  identi¬ 
fy  the  major  characteristics  of  ship¬ 
handling  training  systems,  their 
capabilities  and  limitations,  and 
recommend  specific  levels  of  training 
system  fidelity  and  quality  for 
addressing  various  areas  of  deck  offi¬ 
cer  skill.  A  simulator  functional 
design  specification  and  application 
guidelines  were  also  developed  for 
maritime  academy  cadet  training.  This 
document  identifies  cadet/third  mate 
skills  amenable  to  simulator-based 
training,  recommends  a  specific  train¬ 
ing  system  design  for  achieving  a  set 
of  cadet  training  objectives,  and  pos¬ 
sible  detailed  guidance  information 
for  flexibly  integrating  simulator- 
based  training  into  academy  curricula 
as  desired. 

1.2  PROJECT  GOAL 

The  goal  of  CAORF 's  Maritime  Pilot 
Training  Investigation  is  to  assist 
the  responsible  parties  in  determining 
the  appropriate  role  of  the  ship 
bridge/shiphandling  simulator  within 
the  training  process  of  marine 
pilots.  It  was  the  purpose  of  the 
experiment  reported  herein  to  extend 
the  empirical  investigation  of  simula¬ 
tor-based  training  to  pilots.  The 
techniques  developed  during  the 
earlier  research  in  this  program  were 
tailored  for  this  investigation.  The 
primary  focus  of  the  research  was  on 
the  effectiveness  of  simulator-based 
training  of  experienced  pilots. 
Secondary  issues  addressed  were  the 
effectiveness  of  such  training  for 
different  areas  of  piloting  skill;  the 
effectiveness  of  such  training  as  a 
function  of  the  level  of  pilot  experl- 


ence;  and,  the  opinions  of  simulator- 
wise  experienced  pilots  regarding  the 
potential  of  such  training.  An 
experiment,  was  conducted  at  CAORF  to 
investigate  these  issues.  Its  purpose 
was  to  generate  objective  information 
addressing  the  effectiveness  of  pilot 
training  on  shiphandling  simulators. 
This  experiment  is  the  subject  of  this 
report.  Another  major  product  of  this 
effort  was  the  development  of  guide¬ 
lines  addressing  the  design  and  use  of 
the  simulator-based  shiphandling 
training  system  by  pilots  (Gynther, 
Hammell  and  Pittsley,  1984).  These 
guidelines  have  been  developed  to  be 
used  by  pilots  when  deciding  whether 
or  not  to  use  simulator-based  train¬ 
ing,  when  evaluating  different  avail¬ 
able  training  systems,  and  when  devel¬ 
oping  or  using  a  simulator-based 
training  program. 

1.3  EXPERIMENTAL  OBJECTIVES 

The  specific  issues  that  were  investi¬ 
gated  during  the  experimental  training 


program  tor  pilots,  which  was  con¬ 
ducted  on  the  CAORF  simulator,  were: 

•  Can  a  shiphandling  simulator  be 
used  effectively  to  assist  marine 
pilots  in  acquiring  selected  skills? 

•  What  types  of  benefits  should 
pilots  anticipate  from  simulator- 
based  training  in  specific  skill 
areas? 

•  How  do  these  benefits  from  simula¬ 
tor-based  training  vary  with  the 
experience  level  of  the  pilot? 


These  are  major  issues  for  which  this 
experiment  could  develop  objective 
data.  Although  it  was  beyond  the 
scope  of  this  experiment  to  comprehen¬ 
sively  address  each  of  these  issues, 
it  was  believed  that  they  could  be 
addressed  to  an  acceptable  level  of 
depth  to  provide  guidance  to  the  pilot 
community  regarding  the  use  of  simula¬ 
tors  for  training. 


CHAPTER  2 


METHODOLOGY 


2.1  PROJECT  APPROACH 

The  approach  followed  in  this  empiri¬ 
cal  investigation  was  similar  to  that 
in  earlier  investigations  of  this 
long-term  program.  It  was  based  on  a 
systematic  approach  to  training,  using 
investigative  techniques  developed  by 
the  Air  Force  and  Navy  (i.e.,  instruc¬ 
tional  systems  development  process, 
systems  approach  to  training,  training 
system  analysis),  and  modified  as  ne¬ 
cessary  for  application  in  the  mari¬ 
time  industry,  and  for  specific  appli¬ 
cation  to  the  issues  of  this  investi¬ 
gation.  A  summary  overview  of  the 
approach  follows. 

First,  the  tasks  that  pilots  perform 
at-sea  were  identified.  This  was 
accomplished  by  the  project  team  after 
a  review  of  the  deck  officer  task 
listing  developed  during  the  first 
phase  of  the  Training  and  Licensing 
project  (Hammell,  Williams,  Grasso, 
and  Evans,  1980),  a  review  of  the 
video  tapes  of  piloting  activities 
originally  obtained  during  the  Huffner 
studies  (Huffner,  1978),  visits  to 
several  pilot  associations,  ship  rides 
in  several  pilotage  areas,  and  discus¬ 
sions  with  numerous  pilots.  These 
tasks  were  then  reviewed  by  one  of  the 
project's  pilot  consultants  and  appro¬ 
priate  alterations  made  to  develop  a 
final  set. 

Next,  the  tasks  were  analyzed  into 
skill  and  knowledge  elements,  which 
were  then  transposed  into  a  listing  of 
training  objectives.  These  training 
objectives  were  then  further  evaluated 


and  allocated  to  the  various  training 
media  normally  available  for  pilot 
training  --  classroom,  at-sea,  small 
vessel,  and  simulator  training.  This 
allocation  was  determined  by  identify¬ 
ing  the  media  that  appeared  to  be  the 
most  cost-effective  means  of  achieving 
each  training  objective.  The  result 
of  this  process  was  a  subset  of  marine 
pilot  training  objectives  that  ap¬ 
peared  to  be  appropriate  for  accom¬ 
plishment  via  simulator-based  train¬ 
ing.  The  skill  areas  identified  for 
which  simulator-based  training  is 
potentially  a  cost-effective  medium 
were: 

•  Emergency  Shiphandling  (mainly  ram¬ 
ming  and  grounding  avoidance  prob¬ 
lems) 

•  Shiphandling 

t  Vessel  Characteristics  (unfamiliar 
vessels) 

•  Pilothouse  Procedures 

•  Advanced  Instrumentation 

•  Restricted  Waters  Navigation 
a  Rules  of  the  Road 

•  Vessel-to-Vessel  Communications 

•  Restricted  Waters  Decisionmaking 
(mainly  reaction  to  collision  situ¬ 
ations) 

This  listing  was  then  further  scruti¬ 
nized  to  identify  representative  skill 


Weak  Interest 


areas  for  evaluation  during  this  CAORF 
experiment.  At  this  point  in  the  ana¬ 
lysis,  the  following  issues  were  con¬ 
sidered:  (a)  the  simulator  capabili¬ 

ties  that  would  normally  be  available 
at  a  commercial  training  facility,  (b) 
the  European  simulator-based  pilot 
training  experiences,  and  (c)  the 
causal  factors  identified  in  various 
published  accident  reports  which 
appear  to  indicate  pilot  skill  areas 
that  may  benefit  from  additional 
training. 

This  list  was  validated  by  comparison 
to  independently  sampled  European 
opinions.  Experienced  German  pilots, 
polled  informally  for  their  interest 
in  simulator  oriented  training, 
expressed  various  degrees  of  interest 
in  the  following  training  topics: 

Greatest  Interest 

•  Restricted  waters  maneuvers  of  con¬ 
cern  to  the  pilots  themselves 

•  Emergency  situations  (including 
equipment  failures)* 

a  Communications  ship  to  ship  and 
ship  to  shore 

Moderate  Interest 

•  Outside  the  simulator  review  of 
basic  theory  and  analyses  of  speci¬ 
fic  relevant  casualties 

•  Advanced  instrumentation 

•  Master-pilot  relationships 

§  Vessel  familiarization  with  special 
vessels 

•  Familiarization  with  significant 
changes  to  port  waterways 


♦Including  what  this  study  defined  as 
decisionmaking 


•  Area  familiarization  (not  involving 
changes  to  the  port) 

Although  the  training  areas  of  inter¬ 
est  are  grouped  and  labeled  different¬ 
ly,  the  interests  are  quite  common. 
Local  area  familiarization  is  of  some¬ 
what  weak  interest  mainly  because 
pilots  feel  that  existing  area  famil¬ 
iarization  training  programs  are  quite 
adequate.  Not  that  some  degree  of 
familiarization  cannot  be  accomplished 
on  a  simulator,  but  rather  that  simu¬ 
lator  training  is  not  fully  adequate 
to  the  task  and  that  at-sea  orienta¬ 
tion  is  adequate.  The  master-pilot 
relationship  has  been  previously  iden¬ 
tified  (Gynther,  Hammell,  Grasso  and 
Pittsley,  1982)  as  a  pre-eminently 
critical  shiphandling  requirement  for 
restricted  waters  navigation.  To 
date,  however,  simulator  training  has 
only  evolved  to  address  masters  and 
pilots  separately. 

A  prototype  training  program  was  de¬ 
veloped  for  experimental  purposes  and 
conducted  via  the  simulator  at  CAORF. 
This  research,  using  the  developed 
training  program,  had  the  dual  role  of 
(1)  generating  objective  data  with 
which  to  further  assess  and  document 
the  potential  use  of  simulator-based 
training  of  pilots  and  (2)  providing  a 
valuable  base  of  data  and  experience 
for  development  of  the  previously 
noted  consumers  guide  relating  to  the 
use  of  simulators  for  training  pilots. 

As  previously  noted,  this  experiment 
investigated  three  major  issues:  (1) 
the  effectiveness  of  simulator-based 
training  of  pilots;  (2)  the  effective¬ 
ness  of  such  training  for  three  dif¬ 
ferent  areas  of  piloting  skill;  and 
(3)  the  effectiveness  of  such  training 
for  two  different  levels  of  piloting 
experience.  Pilots  from  East,  Gulf, 
and  West  Coast  ports  participated  in 
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the  experiment.  Both  test  and  train¬ 
ing  exercises  were  conducted  on  the 
simulator  during  each  of  three  four- 
day  training  programs.  Classroom  ses¬ 
sions  were  interspersed  with  the  simu¬ 
lator  exercises  as  part  of  the  train¬ 
ing  program.  The  effectiveness  of  the 
simulator-based  training  was  deter¬ 
mined  on  the  basis  of  group  perfor¬ 
mance  in  certain  controlled  simulator- 
based  shiphandling  exercises  that 
immediately  preceded  and  followed  seg¬ 
ments  of  the  training  program.  It  was 
assumed  that  the  performance  of  the 
pilots  on  the  simulator  during  both 
the  pretraining  and  posttraining  exer¬ 
cises  would  be  representative  of  their 
performance  during  similar  piloting 
operations  at-sea.  Observations  and 
comments  by  the  pilot  coordinators  and 
pilot  participants  were  also  factored 
into  the  analysis. 

This  section  addresses  the  major  as¬ 
pects  of  methodology  used  during  the 
preparation  and  conduct  of  the  experi¬ 
ment.  The  results  of  the  experiment 
are  documented  and  discussed  in 
Chapters  3  and  4. 

2.2  GEOGRAPHIC  DATA  BASE 

It  has  been  a  common  belief  that  a 
port-specific  geographic  data  base 
would  be  necessary  for  the  simulator- 
based  training  of  pilots.  This  would 
represent  a  very  expensive  approach  if 
such  training  were  to  become  a  reality 
for  a  large  segment  of  the  pilot  popu¬ 
lation.  Furthermore,  the  simple  cost- 
effectiveness  of  simulator-based 
training  would  be  seriously  questioned 
if  it  is  necessary  to  develop  port- 
specific  data  bases  for  the  smaller 
pilot  associations  (e.g.,  15  to  20 
members).  Hence,  a  more  cost-effec¬ 
tive  alternative  to  the  port-specific 
data  base  is  necessary.  Although  it 
is  believed  by  many  individuals,  par¬ 
ticularly  pilots,  that  port-specific 
data  bases  are  likely  to  yield  the 


most  effective  pilot  training,  it  is 
necessary  to  consider  training  in  a 
generic-port  data  base  due  to  cost 
considerations. 

The  viability  of  pilot  training  in  a 
generic  port  data  base  was,  therefore, 
a  major  factor  in  this  experiment. 
The  effectiveness  of  simulator-based 
training  for  pilots  was  investigated 
in  a  generic  port  data  base  developed 
specifically  for  this  experiment. 
This  geographic  data  base  was  a  hypo¬ 
thetical  port  specifically  designed  to 
assist  in  the  refinement  of  the  skills 
selected  for  each  training  module.  A 
chart  of  this  hypothetical  port, 
Lotsedorf,  is  contained  in  Figure  1. 
Both  container  and  petroleum  bulk 
cargoes  are  handled  in  the  port.  A 
container  terminal  is  located  at 
Lotsedorf.  The  petroleum  terminal  is 
located  further  up  the  Ball  River.  An 
anchorage  is  located  in  Burton  Bay  for 
vessels  waiting  for  a  berth  at  either 
the  cargo  terminal  or  the  oil  termi¬ 
nal.  The  topography  is  barren  and 
rocky.  When  approaching  from  seaward, 
passage  must  be  made  via  a  winding 
800-foot-wide  channel  through  danger¬ 
ous  shoal  areas.  The  channel  is 
buoyed;  however,  ice  and  frequent 
storms  may  disperse  these  aids  such 
that  mariners  should  navigate  with 
extreme  caution.  The  beacons  on 
Shea's  Ledge,  North  Ledge,  and  South 
Ledge  also  provide  fixed  frequency 
racon  coverage. 

There  are  also  several  potential  bene¬ 
fits  for  conducting  pilot  training  in 
generic  data  bases.  First,  pilots  may 
be  more  receptive  to  generic  data  base 
training  because  they  can  participate 
in  training  without  (a)  focusing  their 
attention  on  the  "inadequacies"  of  the 
training  device  for  simulating  their 
specific  port,  or  (b)  perceiving  that 
they  are  putting  their  credentials  for 
pilotage  in  their  port  on  the  line  if 
they  should  make  a  mistake.  Second, 
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Figure  1.  Approach  to  Lotserdorf  and  Ball  River 


the  utilization  of  generic  data  bases 
for  pilot  training  would  probably  also 
mean  that  a  given  simulator  class 
would  involve  pilots  from  several  geo¬ 
graphic  locales.  A  valuable  cross¬ 
fertilization  of  expertise  may  take 
place,  particularly  for  experienced 
pilots,  since  it  would  potentially 
allow  the  exchange  of  information  con¬ 
cerning  a  variety  of  successful  pilot¬ 
ing  techniques.  Third,  for  training 
specific  skills,  such  as  advanced 
instrumentation,  a  simple  generic  data 
base  may  focus  student  attention 
better  than  a  data  base  with  specific 
port  cues. 

2.3  EXPERIMENTAL  VARIABLES 

The  principle  variable  investigated  in 
this  experiment  was  the  effectiveness 
of  simulator-based  training  for 
pilots.  This  is  a  bottom-line  issue 
of  concern  to  pilots,  directly  relat¬ 
ing  to  the  cost/benefit  ratio  for 
pilot  training  using  simulators  as  a 
supplement  to  the  existing  pilot 
training  programs.  The  training 
effectiveness  was  addressed  in  terms 
of  the  improvement/decrement  in  cer¬ 
tain  piloting  skills  that  resulted 
from  participation  in  the  training 
program. 

Two  secondary  variables  were  investi¬ 
gated  in  addition  to  the  training 
effectiveness  --  pilot  skill  areas, 
and  pilot  level  of  experience.  Each 
of  these  secondary  variables  was 
addressed  in  terms  of  its  impact  on 
the  effectiveness  of  simulator-based 
training.  These  secondary  variables 
are  discussed  below. 

Pilot  Skill  Areas 

Three  pilot  skill  areas  were  examined 
in  this  experiment:  (a)  advanced 
piloting  instrumentation,  (b)  emergen¬ 
cy  shiphandling  and  (c)  restricted 
waters  decisionmaking.  The  first 


skill  area,  advanced  piloting  instru¬ 
mentation,  involved  refining  a  pilot's 
ability  to  effectively  employ  some  of 
the  more  sophisticated  electronic 
navigational  equipment  that  are 
appearing  more  frequently  on  today's 
ships.  Pilots,  particularly  experi¬ 
enced  pilots,  are  well -versed  with  the 
equipment  that  they  employ  during 
their  transits.  However,  several 
independent  sources  have  indicated 
that  pilots  may  benefit  significantly 
from  additional  training  in  utilizing 
various  modern  electronic  equipments, 
such  as  stabilized  radar,  racons, 
doppler  speed  log,  and  rate-of-turn 
indicator.  Skills  related  to  the  use 
of  advanced  instrumentation  have  been 
a  major  area  of  simulator-based  train¬ 
ing  for  pilots  at  the  Wageningen  (the 
Netherlands)  and  Bremen  (Germany) 
training  facilities. 

The  second  skill  area,  emergency  ship¬ 
handling  has  been  generally  recognized 
as  an  area  for  potentially  effective 
training  on  a  simulator.  Most  pilots 
are  receptive  to  this  type  of  simula¬ 
tor-based  training  since  it  obviously 
can  not  be  trained  effectively  at  sea 
due  to  the  high  risks  involved.  The 
piloting  skills  associated  with  keep¬ 
ing  a  large  vessel  under  control  when 
executing  a  crash  stop  in  a  narrow 
channel,  or  turning  a  large  vessel 
around  within  a  confined  channel  with¬ 
out  tugs,  although  infrequently  used, 
are  well -recognized  as  important  and 
desirable  by  many  pilots.  This  type 
of  training  on  a  simulator  is  also 
currently  provided  to  European  pilots. 

The  third  and  final  pilot  skill  area 
investigated  was  decisionmaking  in 
emergency  or  unusual  restricted  waters 
situations.  Discussions  with  many 
pilots  have  indicated  the  importance 
of  being  able  to  rapidly  respond  to 
common  unforeseen  difficult  problems, 
such  as  a  vessel  unexpectedly  depart¬ 
ing  from  an  anchorage  and  crossing 


ownship's  bow.  These  types  of  situa¬ 
tions  often  involve  the  rapid  detec¬ 
tion  of  a  problem,  the  assessment  of 
alternative  actions  available,  the 
selection  of  a  course  of  action,  and 
the  effective  implementation  of  that 
action.  The  use  of  a  simulator  to 
train  this  area  of  pilot  skills 
appears  especially  suited  to  pilots 
with  a  relatively  limited  amount  of 
experience,  since  it  apparently  takes 
many  years  of  experience  to  encounter 
the  range  of  potential  problems  and  to 
develop  the  skills  necessary  for  early 
detection  and  handling  of  these  situa¬ 
tions.  The  simulator  has  potential 
for  this  area  of  training  since  it  can 
be  used  to  provide  a  pilot  with  a  re¬ 
presentative  set  of  problem  situations 
in  a  relatively  short  time,  which 
otherwise  might  require  many  years  of 
experience.  Also,  it  should  also  be 
noted  that  the  risk  of  such  training 
would  be  substantially  less  than  at 
sea. 

Pilot  Experience 

The  effectiveness  of  simulator-based 
training  is  likely  to  depend  on  many 
factors.  Pilot  skill  area  is,  per¬ 
haps,  the  most  relevant  of  these  fac¬ 
tors.  Another  important  factor  may  be 
the  level  of  pilot  experience.  Simu¬ 
lator-based  training  may  be  more 
effective  for  highly  experienced 
pilots  regarding  certain  skill  areas, 
and  perhaps  more  effective  for  pilots 
with  limited  experience  regarding 
other  skill  areas.  Two  levels  of 
pilot  experience  were  examined  regard¬ 
ing  the  effectiveness  of  simulator- 
based  training:  (a)  limited  experience 
(less  than  5  years  as  a  licensed 
pilot),  and  (b)  extensive  experience 
(greater  than  10  years  as  a  licensed 
pilot);  see  Pilot  Participants  (Para¬ 
graph  2.5)  for  additional  information 
regarding  the  qualifications  of  parti¬ 
cipating  pilots.  Apprentice  pilots 
were  not  considered  due  to  their  gen¬ 


eral  unavailability.  As  a  result  of 
the  reduced  shipping  in  recent  years, 
many  pilot  associations  do  not  pre¬ 
sently  have  apprentices,  nor  do  they 
anticipate  accepting  apprentices  for 
several  years. 

2.4  EXPERIMENTAL  DESIGN 

The  experimental  design  for  this 
experiment  is  summarized  in  Figure  2. 
A  1-week  experimental  training  program 
was  provided  to  three  groups  of 
pilots,  one  group  each  week.  Each  of 
these  groups  were  composed  of  four 
experienced  pilots.  Within  each  group 
there  was  a  mix  of  limited  experience 
and  extensive  experience  pilots.  Each 
1-week  training  program  was  divided 
into  three  independent  training 
modules,  with  each  module  addressing 
one  of  the  pilot  skill  areas  discussed 
above.  The  objective  data  used  for 
training  effectiveness  evaluations 
were  based  on  pre  and  postexercises 
for  each  pilot,  individually  conducted 
before  and  after  each  training 
module.  Simulator  difficulties  pre¬ 
vented  obtaining  data  for  the  third 
training  module  within  the  experiment¬ 
al  program.  As  a  result,  the  objec¬ 
tive  data  derived  from  shiphandling 
performance  on  the  simulator  exercises 
were  available  for  training  Modules  I 
and  II.  Pilot  observations  and  opin¬ 
ion  data  were  collected  for  each 
module,  including  some  data  for  the 
third  training  module.  Chapters  3  and 
4  contain  a  detailed  presentation  and 
discussion  of  the  results  obtained 
during  this  experimental  program. 

2.5  PILOT  PARTICIPANTS 

Twelve  licensed  pilots  participated  in 
this  experimental  CAORF  program. 
These  pilots  were  equally  distributed 
from  East  Coast,  Gulf  Coast,  and  West 
Coast  ports  (i.e.,  four  pilots  from 
each  coast).  The  following  organiza¬ 
tions  provided  participants: 


WEEK  1 


Figure  2.  Experimental  Design  Summary 


United  New  York  New  Jersey  Sandy 
Hook  Pilot  Association 


Pilot  Association  for  the  Bay  and 
River  Delaware 


Charleston  (S.C.)  Branch  Pilot  As¬ 
sociation 


t 


Tampa  Bay  Pilots 


Cresent  River 
Orleans) 


Port  Pilots  (New 


Corpus  Christi  Pilots 

San  Francisco  Bar  Pilots 

Jacobsen  Pilot  Service  (Long  Beach) 


The  average  experience  across  all  the 
participants  was  9.0  years  as  a  li¬ 
censed  pilot.  The  range  of  this 
experience  was  from  1.5  years  to  23.0 
years.  There  were  five  pilots  with 
less  than  5.0  years  experience  and 
seven  pilots  with  greater  than  10.0 


years  experience.  The  average  age  of 
these  individuals  was  39.8  years,  with 
their  ages  varying  from  26  to  56 
years.  Ten  of  the  twelve  pilots  were 
between  30  and  50  years  old.  None  of 
the  pilots  had  previously  participated 
in  a  training  program  on  an  electronic 
ship  bridge/shiphandling  simulator, 
although  several  had  previously  been 
to  CA0RF  as  test  subjects  during  other 
research  experiments.  One  pilot  had 
participated  in  the  simulator  training 
program  at  Grenoble,  France  which 
employs  physical  ship  models. 


2.6  TRAINING  PROGRAM 


The  prototype  simulator-based  training 
program  for  pilots  that  was  conducted 
at  CA0RF  as  part  of  this  experiment 
addressed  only  a  subset  of  the  pilot 
skills  that  analysis  identified  as 
potentially  appropriate  for  develop¬ 
ment/improvement  via  simulator-based 
training.  Limited  resources  prohib¬ 
ited  the  investigation  of  additional 
skills.  The  reader  should  note  that 
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this  experimental  training  program  was 
developed  specifically  for  this  ex¬ 
periment,  and  thus  is  not  necessarily 
a  training  program  that  should  be  of¬ 
fered  to  pilots  at  a  commercial  train¬ 
ing  facility.  As  an  experimental 
training  program  it  was  designed  to 
cover  several  skill  areas  in  order  to 
provide  an  indication  of  the  possible 
benefits  from  simulator-based  train¬ 
ing.  It  was  not  designed  to  compre¬ 
hensively  address  these  skills  to  the 
level  of  refinement  that  might  be 
accomplished  at  a  simulator  facility 
dedicated  to  training. 


The  experimental  training  program  was 
a  1-week  program  designed  for  a  class 
of  four  pilots.  It  was  composed  of 
three  training  modules,  which  were 
based  on  the  three  skill  areas  identi¬ 
fied  and  discussed  in  Paragraph  2.3. 
These  three  modules  were  (a)  Advanced 
Piloting  Instrumentation,  (b)  Emergen¬ 
cy  Shiphandling,  and  (c)  Restricted 
Waters  Decisionmaking.  The  specific 
training  objectives  developed  for  each 
module  are  presented  in  Table  1. 


The  structure  of  the  training  program 
involved  a  combination  of  classroom 
and  simulator  training.  It  is  out¬ 
lined  in  Figure  3.  The  training  stra¬ 
tegy  used  in  the  program  was  a  modifi¬ 
cation  of  seminar,  role  playing,  and 
case  study  techniques,  which  were  con¬ 
sidered  appropriate  for  skilled  pro¬ 
fessionals  focusing  on  cognitive 
tasks.  Each  training  module  commenced 
with  the  trainees  individually  han¬ 
dling  the  ownship  in  a  controlled 
simulator  exercise.  This  exercise  was 
then  discussed  during  the  classroom 
session  as  a  case  study  example  of  a 
situation  requiring  particular  skills, 
which  would  be  addressed  during  the 
training  module.  After  the  classroom 
session,  the  pilots  participated  in  a 
series  of  exercises  on  the  simulator. 
They  applied  and  practiced  the  princi¬ 
ples  and  techniques  discussed  during 


TABLE  1.  TRAINING  OBJECTIVES  FOR 
PROTOTYPE  SIMULATOR  TRAINING  PROGRAM 
FOR  MARINE  PILOTS 


Training  Module  I:  Advanced  Piloting 
Instrumentation 


•  The  trainee  shall  be  able  to  effec¬ 
tively  execute  a  constant  radius 
turn  under  various  operational  con¬ 
ditions  while  employing  a  rate-of- 
turn  indicator,  a  doppler  speed 
log,  and  racons. 


Training  Module  II:  Emergency  Ship¬ 
handling  Skills 


•  The  trainee  shall  be  able  to  safely 
turn  an  80,000  dwt  tanker  around 
within  a  confined  channel  under 
various  operational  conditions. 


Training  Module  III:  Decisionmaking 
in  Unusual  or  Emergency  Restricted 
Waters  Situations 


The  trainee  shall  be  able  to  effec¬ 
tively  respond  to  an  unanticipated 
vessel  departing  from  an  anchorage 
or  berth  within  a  confined  channel 
under  various  operational 
conditions. 


the  classroom  period.  During  this 
group  exercise  period,  the  trainees 
were  rotated  between  the  positions  of 
pilot,  helmsman,  and  observer. 


The  training  program  was  coordinated 
by  two  highly  experienced  and  re¬ 
spected  pilots  currently  active  in 
different  pilot  associations.  Each  of 
these  coordinators  are  articulate  in¬ 
dividuals  as  regards  piloting  and 
piloting  techniques.  They  possessed 
previous  experience  with  shiphandling 
simulators,  and  were  versed  with  many 
of  the  research  protocols  employed  at 
CAORF.  As  coordinators  they  were  the 
primary  interface  with  the  pilot  par- 
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ticipants,  providing  technical  infor¬ 
mation,  overseeing  the  various  train¬ 
ing  activities,  and  directing  the 
training  program.  They  also  assisted 
during  the  development  of  the  experi¬ 
ment  and  the  training  program. 

2.7  EXPERIMENTAL  PROCEDURES 

Extreme  care  was  exercised  in  conduct¬ 
ing  the  training  program  to  minimize 
uncontrolled  aspects  of  the  experiment 
which  can  result  from  variations  in 
the  selection,  training,  and  debrief¬ 
ing  of  subjects.  The  following  proce¬ 
dures  were  followed: 

a.  Upon  initial  contact,  via  his  re¬ 
spective  pilot  association,  the 
pilot  participant  was  appraised  of 
the  experimental  nature  of  the 
training  program.  He  was  told  that 
it  is  anticipated  he  would  receive 
beneficial  training;  however,  his 
primary  purpose  was  to  assist  the 
CAORF  staff  in  the  evaluation  of 
this  type  of  simulator-based  train¬ 
ing  for  pilots. 

b.  Upon  arrival  at  CAORF,  each  pilot 
was  provided  with  additional  infor¬ 
mation  during  the  introductory  ses¬ 
sions  concerning  the  purpose  of  the 
Pilot  Training  Investigation,  the 
objectives  of  this  particular  ex¬ 
perimental  training  program,  and 
his  role  during  the  coming  week. 
He  was  also  provided  with  an  appro¬ 
priate  familiarization  of  the  CAORF 
simulator  including  an  opportunity 
to  handle  the  simulated  vessel  in 
the  generic  data  base  used  during 
the  program. 

c.  During  testing  periods,  each  pilot 
was  on  the  simulator  bridge  alone 
except  for  a  helmsman,  who  was  a 
member  of  the  CAORF  staff.  The 
pilot  was  provided  with  appropriate 
instructions  prior  to  the  commence¬ 
ment  of  each  test  exercise.  After 


each  test  the  exercise,  he  was  in¬ 
structed  to  refrain  from  discussing 
the  exercise  with  other  pilots 
until  the  next  class  session.  A 
variety  of  pertinent  maritime  films 
were  available  for  viewing  by  the 
pilots  when  they  were  not  on  the 
simulator  during  the  testing  peri¬ 
ods. 

d.  During  the  training  periods,  the 
pilots  were  rotated  through  the 
position  of  pilot  and  helmsman. 
During  Modules  I  and  II  those 
pilots  not  participating  in  the 
exercise  were  located  on  the  bridge 
in  an  unobtrusive  location  with  the 
coordinator.  During  Module  III, 
the  most  advantageous  position  for 
the  off-watch  pilots  and  the  in¬ 
structor  was  determined  to  be  at 
the  human  factors  monitoring  sta¬ 
tion,  where  they  could  anticipate, 
observe,  and  discuss  exercise 
events  without  disturbing  those 
pilots  handling  the  simulated  ves¬ 
sel. 

e.  Upon  completion  of  the  training, 
each  pilot  was  individually  de¬ 
briefed  by  a  member  of  the  research 
staff,  who  completed  the  debriefing 
questionnaire  contained  in  Appendix 
A. 


2.8  OWNSHIP  CHARACTERISTICS 

The  following  are  characteristics  of 
the  simulated  ship  that  was  used  dur¬ 
ing  the  experimental  training  program 
at  CAORF: 

t  Type:  80,000  dwt  tanker 

•  Length  overall  (LOA):  800  feet 

•  Length  between  perpendiculars 

(LBP) :  763  feet 

•  Location  of  pilothouse:  Aft 


•  Beam:  125  feet 


•  Draft  (full  load): 

-  Fwd  -  39.9  feet 

-  Aft  -  39.9  feet 

•  Propulsion: 

-  Slow  speed  diesel 

-  23,000  horsepower 

-  Single  screw 

-  Direct  pilothouse  control 


2.9  TEST  EXERCISES 

Four  test  exercises  were  developed  for 
the  experimental  training  program. 
The  purpose  of  these  test  exercises 
was  to  generate  the  objective  pilot 
performance  data  used  to  assess  the 
gain/loss  in  performance  that  resulted 
from  each  training  module.  Test  exer¬ 
cise  1  was  designed  to  evaluate 
trainee  performance  prior  to  Training 
Module  I.  Test  exercise  2  was  de¬ 
signed  to  provide  both  (a)  posttrain¬ 
ing  performance  data  for  Module  I,  and 
(b)  pretraining  performance  data  for 
Module  II.  Similarly,  test  exercise  3 
was  designed  to  provide  (a)  posttrain¬ 
ing  data  for  Module  II,  and  (b)  pre¬ 
training  data  for  Module  III.  Final¬ 
ly,  test  exercise  4  was  designed  to 
provide  posttraining  performance  data 
for  Module  III.  Due  to  simulator  dif¬ 
ficulties,  which  affected  the  training 
program  schedule,  test  exercise  4  was 
not  accomplished.  As  a  result,  the 
objective  performance  data  required  to 
evaluate  the  training  effectiveness  of 
Module  III  was  not  available.  The 
subjective  evaluation  accomplished  at 
the  end  of  the  program,  however,  was 
conducted  for  Module  III  using  two 
groups  of  pilots.  The  following  is  a 
brief  summary  of  each  of  the  three 
test  exercises  that  were  accomplished 
and  utilized  as  the  basis  for  the  ob¬ 
jective  analysis  described  in  Chapters 
3  and  4. 


•  Test  Exercise  1  -  The  80,000  dwt 

tanker  (fully  loaded)  was  posi¬ 
tioned  inbound  on  the  centerline  of 
Channel  C  at  position  A  in  Figure 
1.  Initial  speed  was  8  knots.  It 
was  a  clear  day  with  unrestricted 
visibility.  A  1.0  knot  current  was 
setting  towards  090°.  Buoys  C8 

and  CIO  were  missing  as  a  result  of 
a  recent  storm.  The  trainee  was 

directed  to  safely  negotiate  the 
1.0  nm  radius  turn  between  Channel 
C  and  Channel  D  utilizing  the 
available  aids  to  navigation.  This 
test  exercise  was  designed  to  pro¬ 
vide  pilot  performance  data  prior 
to  Training  Module  I. 

•  Test  Exercise  2  -  The  80,000  dwt 

tanker  (fully  loaded)  was  posi¬ 
tioned  again  inbound  on  the  center- 
line  of  Channel  C  at  position  A. 
Initial  speed  was  8  knots.  Visi¬ 
bility  was  unrestricted,  and  a  1.0 
knot  current  was  again  setting 
towards  090°.  As  in  test  exer¬ 
cise  1,  buoys  C8  and  CIO  were  miss¬ 
ing.  The  trainee  was  directed  to 
safely  negotiate  the  1.0  nm  radius 
turn  between  Channel  C  and  Channel 
D  utilizing  the  available  aids  to 
navigation.  Upon  completion  of  the 
turn  and  entry  in  Channel  D,  the 
pilot  was  told  that  he  should  turn 
the  ship  around  in  Channel  D  with¬ 
out  tugs  since  there  had  been  an 
accident  which  was  obstructing  the 
channel  at  buoy  N19.  The  1.0  nm 
radius  turn  was  designed  to  provide 
pilot  performance  data  after  Module 
I,  while  the  turn  around  was  de¬ 
signed  to  provide  performance  data 
prior  to  Module  II. 

•  Test  Exercise  3  -  The  80,000  dwt 
tanker  (fully  loaded)  was  posi¬ 
tioned  outbound  on  the  centerline 
of  Channel  D  at  position  B  in  Fi¬ 
gure  1.  Initial  speed  was  8 
knots.  Visibility  was  unrestricted 


and  a  1.0  knot  current  was  setting 
toward  270°.  The  trainee  was 
told  that  he  should  turn  the  ship 
around  in  Channel  D  without  tugs 
since  there  has  been  an  accident 
which  is  obstructing  the  channel  at 
buoy  N7. 

Once  the  ship  is  turned  around,  the 
pilot  was  directed  to  -eturn  to  his 
berth  at  the  oil  terminal.  As  own- 
ship  proceeded  inbound  the  visibil¬ 
ity  was  gradually  reduced  to  ap¬ 
proximately  0.5  nm.  As  ownship 
approached  the  container  terminal, 
a  containership  unexpectedly  backed 
into  the  channel  less  than  1.0  nm 
ahead.  The  containership  desired 
to  proceed  outbound.  The  earlier 
turn  around  within  Channel  0  was 
designed  to  provide  post-Module  II 
performance  data,  while  the  unanti¬ 
cipated  backing  of  the  container- 
ship  was  designed  to  provide  per¬ 
formance  data  regarding  pilot  de¬ 
cisionmaking  skills  prior  to  Module 
III. 

2.10  PERFORMANCE  MEASURES 

Two  major  sources  of  data  were  used  in 
the  experiment:  (1)  objective  perfor¬ 
mance  data  collected  during  the  test 
scenarios;  and  (2)  subjective  observa¬ 
tions  and  opinions  of  the  pilot  parti¬ 
cipants,  collected  during  the  debrief¬ 
ing  at  the  end  of  the  training  pro¬ 
gram.  The  subjective  data  are  com¬ 
prised  of  straightforward  pilot  opin¬ 
ions  concerning  the  specific  issues 
addressed.  Their  analysis  is  in  Para¬ 
graph  4.4  in  terms  of  the  proportion 
of  various  opinions  expressed  regard¬ 
ing  each  issue/question.  The  objec¬ 
tive  data,  on  the  other  hand,  were 
comprised  of  various  ownship  and  situ¬ 
ation  parameters,  some  of  which  were 
automatically  recorded  by  the  simula¬ 
tor/computer  and  some  which  were  manu¬ 
ally  recorded  by  an  observer.  In  both 
cases,  these  objective  data  were 


transformed  into  performance  measures 
relevant  to  the  particular  training 
objectives,  and  then  statistically 
analyzed  to  evaluate  training  effec- 
tivem  ss. 

Several  performance  measures  were 
identified  for  the  types  of  pilot 
skills  involved  in  Modules  I  and  II. 
These  performance  measures  were  or¬ 
ganized  into  three  categories  for  each 
module,  based  on  their  relevance  to 
the  training  objectives  of  the  respec¬ 
tive  module:  (a)  primary  performance 
measure,  (b)  secondary  performance 
measures,  and  (c)  third-order  perfor¬ 
mance  measures.  The  primary  perfor¬ 
mance  measure  is  the  single  measure 
most  applicable  to  the  particular 
training  objectives,  pilot  skills,  and 
relevant  aspects  of  shiphandling.  It 
represents  the  bottom-line  with  regard 
to  each  respective  training  program 
module.  Previous  research  of  this 
type  in  the  maritime  field  has  shown, 
however,  that  for  various  reasons  sin¬ 
gle  performance  measures  may  not  be 
insensitive  to  differences  in  perfor¬ 
mance.  Hence,  multiple  performance 
measures  are  necessary  to  understand 
the  processes  involved.  Furthermore, 
these  additional  performance  measures 
address  other  relevant,  although 
secondary  and  third-order,  aspects  of 
performance  and  behavior.  They  assist 
in  analyzing  and  interpreting  the 
experimental  effects  by  providing  data 
regarding  associated  facets  of  perfor¬ 
mance. 

A  description  of  the  performance  mea¬ 
sures  used  with  each  module  follows. 

Module  I  -  Advanced  Instrumentation 

The  focus  of  this  training  program 
module  was  the  effective  utilization 
of  advanced  instrumentation,  namely 
the  rate-of-turn  indicator,  the 
doppler  speed  log,  and  the  available 
racons,  to  execute  a  constant  radius 
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turn.  The  ability  to  achieve  a  con¬ 
stant  radius  turn  using  these  instru¬ 
ments  provides  a  set  of  piloting 
skills  that  allows  considerable  vessel 
control  during  the  navigation  of 
turns;  as  such,  it  represents  one 
effective  piloting  technique,  to  aug¬ 
ment  other  available  techniques. 

•  Primary  Performance  Measure 

-  Sum  of  Distances  Off  1.0  nm  Ra¬ 
dius:  This  performance  measure 
is  the"  sum  of  the  absolute  dif¬ 
ference  in  feet  between  the 
ideal  1.0  nm  radius  and  the 
actual  position  of  the  vessel's 
center  of  gravity,  at  four  dif¬ 
ferent  data  lines  through  the 
turn.  These  data  lines  were 
spaced  at  intervals  of  1/4  of 
the  turn,  DL2  through  DL5  in 
Figure  4.  The  first  data  line 
(DL2)  was  located  at  the  end  of 
the  first  one-fourth  of  the 
turn,  and  the  final  data  line 


placed  at  the  end  of  the  com¬ 
plete  turn.  Ideally,  the  vessel 
should  have  maintained  a  precise 
1.0  nm  radius  from  the  racon 
throughout  the  turn.  This  con¬ 
stant  radius  was  the  objective 
of  training  in  Module  I.  The 
summation  of  this  ownship  posi¬ 
tion  deviation  at  each  of  the 
data  lines  provides  a  relevant 
measure  of  the  degree  to  which 
the  pilot  was  able  to  maintain 
the  constant  radius. 

•  Secondary  Performance  Measure 

-  Sum  of  Differences  Between  Actu¬ 
al  and  Desired  Rate-of-Turn: 
This  measure  is  similar  to  the 
previous  performance  measure  in 
that  it  is  calculated  at  speci¬ 
fic  data  lines  in  the  turn.  To 
achieve  the  constant  1.0  nm  ra¬ 
dius  turn,  the  vessel  had  to 
maintain  a  rate-of-turn  of  ap¬ 
proximately  0.11  deg/sec.  This 
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value,  which  has  both  a  theore¬ 
tical  and  empirical  basis,  as¬ 
sumes  an  average  speed  over  the 
ground  of  approximately  7  knots 
as  the  ship  proceeds  through  the 
turn.  (Note:  the  theoretical 
basis  was  provided  to  the 
trainees  as  part  of  the  class¬ 
room  training.)  Hence,  this 
measure  addresses  the  differ¬ 
ences  between  the  ideal  rate-of- 
turn  and  the  actual  rate-of-turn 
achieved.  The  data  lines  at 
which  the  rates-of-turn  were 
measured  were  identical  to  those 
employed  for  the  previous  mea¬ 
sure,  with  the  exception  that 
the  final  data  line  at  the  end 
of  the  turn  was  not  used  (DL5). 
This  data  line  was  omitted  since 
the  vessel,  having  completed  its 
constant  radius  turn,  should 
have  a  zero  rate  of  turn  at  that 
point. 


•  Third-Order  Performance  Measures 


-  Number  of 

Rudder 

Orders 

—  the 

number  of 

rudder 

orders 

issued 

during  the  exercise 

-  Number  of 

Course 

Orders 

—  the 

number  of 

course 

orders 

issued 

during  the 

exercise 

-  Minimum 

Distance 

to 

Channel 

Boundary 

—the 

closest 

point 

ownship  came  to 

the 

channel 

boundary  during  the  exercise 


Channel  Excursion  --  the  number 
of  pilots  that  had  any  part  of 
their  vessel  extend  outside  the 
channel  boundaries  during  the 
exercise 


Module  II  -  Emergency  Shiphandling 


This  module  emphasized  the  refinement 
of  skills  involved  when  turning  a 
large  tanker  around  within  a  confined 


channel  without  tugs.  The  specific 
test  exercise  involved  turning  an 
80,000  dwt  tanker  around  within  a  1500 
ft  channel  without  tugs,  and  with  a 
1-knot  current  on  the  starboard  bow. 
Although  some  pilots  may  not  consider 
this  exercise  as  emergency  ship¬ 
handling  training,  most  pilots  agreed 
that  it  would  not  normally  be  at¬ 
tempted  except  under  emergency  condi¬ 
tions.  The  Dutch  pilots  during  their 
simulator  training  at  Wageningen  de¬ 
veloped  an  effective  technique  for 
safely  accomplishing  this  maneuver. 
This  technique  was  the  focus  during 
the  emergency  shiphandling  module  of 
the  CAORF  experimental  training  pro¬ 
gram.  The  technique  involves  balanc¬ 
ing  the  cross-channel  component  of  the 
ship's  speed  against  the  cross-channel 
component  of  the  current,  according  to 
Table  2,  as  the  vessel  slows.  The 
pilot  applies  a  course  correction  to 
head  up  into  the  current  based  on  his 
speed  through  the  water  at  appropriate 
intervals  of  time.  For  the  test  exer¬ 
cise,  this  method  is  extremely  helpful 
for  the  first  30  degrees  of  course 
change  (i.e.,  nearly  heading  into  the 
current) . 


•  Primary  Performance  Measure 


-  Channel  Excursions:  This  mea¬ 
sure  Ts  Die  number  of  pilots 
that  had  any  part  of  their  ves¬ 
sel  extend  beyond  the  channel 
boundaries.  As  such,  it  is 
clearly  a  bottom-line  perfor¬ 
mance  measure  for  the  situation 
addressed  in  Module  II.  Channel 
excursions  were  determined 
graphically  by  viewing  geograph¬ 
ic  plots  containing  the  hull 
outline  of  ownship  throughout 
the  exercise. 


•  Secondary  Performance  Measure 


-  Minimum  Distance  to  Channel 
Boundary:  This  measure  repre- 


ssmsm 


TABLE  2.  COURSE  CORRECTION  (DEGREES)  FOR  CROSS-CHANNEL  CURRENT 


COMPONENT  CURRENT XCHANNEL 
(MILES/HOUR) 

SPEED  THROUGH  THE  WATER 
(MILES/HOUR) 

% 

1 

IV, 

2 

2/, 

3 

9 

3 

6 

9 

12 

15 

18 

8 

3’/, 

7 

10/, 

14 

17/, 

21 

7 

4 

8 

12 

16 

20’/, 

25 

6 

5 

10 

15 

20 

25 

30 

5 

6 

12 

18 

24 

30 

36 

4 

7V> 

10 

221, 

30 

37/, 

40 

3 

10 

20 

30 

42 

57 

90 

2 v, 

12 

24 

36 

53 

SO 

2 

15 

30 

40 

90 

1  Vj 

20 

42 

90 

1 

30 

90 

0 

EXAMPLE: 

Cross-Channel  Current  Component  =  1/2  knot 
Speed  Through  The  Water  =  8  knots 
Course  Correction  =  3-1/2° 


Source:  J.  Jussem  and  DeBoer,  "Seven  Years  Experience  With  Simulator  Training  at 
VLCC  Pilots  in  the  Netherlands,"  Proceedings  Second  International  Conference  on 
Marine  Simulation.  National  Maritime  Research  Center,  Kings  Point,  New  York, 
1981. 


sents  the  single  point  at  which 
ownship  was  closest  to  the  chan¬ 
nel  boundary.  This  distance  was 
measured  from  the  skin  of  the 
ship,  not  the  center  of  gravity, 
to  the  channel  boundary.  The 
units  of  measure  were  "feet". 
If  the  vessel  went  beyond  the 
channel  boundary,  a  zero  value 
was  entered. 

t  Third-Order  Performance  Measures 

-  Course  correction  at  4  knots  — 
the  difference  between  ownships 
course  and  the  channel  heading 
when  ownship' s  speed  was  reduced 
to  4  knots. 


-  Number  of  Rudder  Orders  prior  to 
30°  heading  change 

-  Number  of  Course  Orders  prior  to 
30°  heading  change 

-  Number  of  Engine  Orders  prior  to 
30°  heading  change 

These  third-order  performance  measures 
are  closely  related  to  the  Dutch  turn 
around  technique,  which  was  emphasized 
during  the  experimental  training  pro¬ 
gram.  Four  knots  was  considered  as  an 
appropriate  sample  point  to  investi¬ 
gate  the  pilot's  adherence  to  the  re¬ 
commended  course  correction  provided 
in  Table  2  as  he  slows  the  vessel. 


For  the  rudder,  course,  and  engine  as  a  result  of  the  training  program, 

orders,  an  upper  limit  of  30  degrees  (pre  versus  posttest  performance) 

heading  change  was  chosen  because,  for  yielding  multiple  measures  of  training 

the  particular  test  exercises  em-  effectiveness.  These  measures  of 
ployed,  this  value  appeared  to  define  change  in  performance  (i.e.,  between 

the  limit  of  the  critical  region  where  the  two  tests  for  each  module)  are  the 

the  Dutch  technique  was  most  helpful.  basis  for  the  findings  presented  and 

discussed  in  the  subsequent  Results 
The  data  analysis  focused  on  changes  and  Discussion  chapters, 

in  each  of  these  performance  measures 


i 


CHAPTER  3 


RESULTS 


This  section  presents  the  results  ob¬ 
tained  from  the  CAORF  experiment,  in 
terms  of  the  data  collected  for,  and 
significance  of,  each  of  the  perfor¬ 
mance  measures.  The  evidence  is  sum¬ 
marized  herein,  but  is  not  discussed. 
The  discussion  of  the  results,  ad¬ 
dressing  each  of  the  issues  under  in¬ 
vestigation,  is  contained  in  Chapter 
4.  The  questionnaire  data  collected 
from  each  of  the  pilots/subjects  are 
presented  and  discussed  in  the  Discus¬ 
sion  chapter. 

The  results  are  segmented  under  each 
of  the  two  training  modules  —  Ad¬ 
vanced  Instrumentation,  and  Emergency 
Shiphandling. 


3.1  ADVANCED  INSTRUMENTATION  — 

MODULE  I 

The  results  pertaining  to  this  module 
are  presented  below  regarding  Training 
Effectiveness  and  Pilot  Experience. 

Training  Effectiveness 

The  training  emphasis  in  Module  I  fo¬ 
cused  on  maneuvering  a  vessel  around  a 
constant  radius  turn,  such  that  the 
ship's  track  would  maintain  an  appro¬ 
priate  constant  radius  through  the 
turn.  The  constant  radius  of  the  turn 
used  during  the  test  was  1  nautical 
mile  (nm).  Hence,  ideally,  a  1  nm 
mile  radius  would  have  been  achieved 
during  the  test  scenarios.  The  sta¬ 
tistical  data  pertaining  to  this  per¬ 
formance  measure,  and  five  additional 
performance  measures,  are  contained  in 


Table  3.  The  primary  performance  mea¬ 
sure  is  the  sum  of  the  difference  be¬ 
tween  the  actual  ship's  position  and 
the  1  nm  radius  at  each  of  four  data 
lines  throughout  the  turn  (i.e.,  see 
Figure  4,  and  the  explanation  in 
Chapter  2).  This  primary  performance 
measure,  sum  of  distance  off  1  nm  ra¬ 
dius,  was  reduced  from  a  mean  of  734.0 
to  384.9  (p^0.025),  showing  a  signi¬ 
ficant  gain  in  pilot  performance. 
This  finding  demonstrates  the  poten¬ 
tial  effectiveness  of  advanced  instru¬ 
mentation  training  for  pilots  on  a 
simulator. 

The  secondary  performance  measure  in 
Module  1  addresses  the  actual  rate-of- 
turn  maintained  in  comparison  with  the 
ideal  rate-of-turn  (0.11  degrees  per 
minute).  Pilot  performance  regarding 
this  measure  was  also  found  to  signi¬ 
ficantly  increase  as  a  result  of  the 
training  program.  The  mean  error  in 
rate-of-turn  was  reduced  from  0.28  to 
0.14  (p^O.OOl)  as  a  result  of  the 
training  program.  Whereas  the  sum  of 
distance  off  the  1  nm  radius  repre¬ 
sents  the  bottom-line  of  performance 
in  this  module,  the  sum  of  difference 
between  the  actual  rate-of-turn  and 
the  desired  rate-of-turn  represents 
the  behavior  of  pilots  in  achieving 
that  performance. 

Two  additional  performance  measures 
describing  specific  changes  in  pilot¬ 
ing  behavior  were  also  found  to  change 
significantly  as  a  result  of  this 
training  module  (see  Table  3).  The 
mean  minimum  distance  to  channel 
boundary  increased  significantly  from 


OVERALL  PILOT  PRETRAINING  VERSUS  POSTTRAINING  COMPARISON 
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143  feet  to  237.5  feet  (p <0.025) . 
This  finding,  of  course,  would  be 
expected  to  correlate  highly  with  the 
primary  performance  measure,  sum  of 
the  distance  off  1  nm  radius.  The 
number  of  course  orders  issued  by  the 
pilots  were  reduced  at  a  marginally 
significant  level  as  a  result  of  the 
training  program,  from  a  mean  of  2.25 
to  1.17  (p<0.10). 

The  remaining  third  order  performance 
measures  did  not  yield  a  significant 
change  in  performance  as  a  result  of 
this  training  module.  The  number  and 
strength  of  significant  results,  how¬ 
ever,  ( i . e . ,  four  of  the  six  perfor¬ 
mance  measures,  including  the  primary 
and  secondary  measures)  strongly  indi¬ 
cates  that  effective  pilot  training 
was  achieved  in  the  advanced  instru¬ 
mentation  module. 

Pilot  Experience 

The  results  pertaining  to  the  impact 
of  experience  on  the  effectiveness  of 
the  advanced  instrumentation  training 
module  are  summarized  in  Tables  4  and 
5,  for  the  limited  and  extensive  ex¬ 
perienced  groups. 

Limited  Experience.  The  primary 
performance  measure,  sum  of  distance 
off  1  nm  radius,  did  not  change  signi¬ 
ficantly,  even  though  this  mean  per¬ 
formance  improved  substantially  be¬ 
tween  the  pretest  and  posttest  (626.5 
feet  to  447.2  feet).  The  secondary 
performance  measure,  sum  of  the  dif¬ 
ferences  between  actual  rate-of-turn 
and  desired  rate-of-turn,  did  yield  a 
significant  change  in  performance, 
with  a  reduction  in  the  mean  error 
from  0.24  to  0.13  degree  per  second 
(p<0.05).  The  behavior  of  the  li¬ 
mited  experience  group  in  bringing  the 
vessel  around  the  constant  radius 
turn,  therefore,  was  found  to  improve 
significantly  as  a  result  of  the 
training  module,  even  though  this  did 


not  result  in  a  significant  change  in 
the  bottom-line  vessel  performance. 

The  limited  experience  group  did  not 
show  a  significant  change  in  perfor¬ 
mance  on  the  remaining  performance 
measures.  In  summary,  therefore,  a 
positive,  although  weak,  indication  of 
training  effectiveness  was  observed 
for  the  limited  experienced  group  of 
pilots  with  regard  to  advanced  instru¬ 
mentation  training. 


Extensive  Experience.  The  effec¬ 
tiveness  of  advanced  instrumentation 
training  for  the  pilots  with  extensive 
experience  was  found  to  be  strong, 
contrary  to  that  observed  for  the 
pilots  with  limited  experience,  with 
several  performance  measures  showing 
significant  improvement  (Table  5). 
The  sum  of  distance  off  1.0  nm  radius, 
the  primary  performance  measure, 
showed  a  significant  increase  in  per¬ 
formance  with  the  mean  sum  of  dis¬ 
tances  of  1.0  nm  radius  substantially 
reduced  from  821.1  feet  to  343.2  feet 
( p  <  0. 05 )  as  a  result  of  training. 
The  sum  of  the  differences  between  the 
actual  rate-of-turn  and  the  desired 
rate-of-turn,  the  secondary  perfor¬ 
mance  measure,  was  also  found  to  show 
a  significant  improvement,  with  the 
rate-of-turn  error  reduced  from  0.32 
degree  per  second  to  0.14  degree  per 
second  (p <0.005). 

Two  of  the  four  third  order  perfor¬ 
mance  measures  were  also  found  to  show 
a  significant  change  in  performance. 
The  minimum  distance  to  channel 
boundary  increased  from  109.5  feet  to 
250.0  feet  (p<0.025).  The  number  of 
channel  excursions  was  significantly 
reduced  from  4  on  the  pretest  to  zero 
on  the  posttest  (p<0.06).  These  re¬ 
sults  indicate  substantial  improvement 
for  the  pilots  with  extensive  experi¬ 
ence  as  a  result  of  the  advanced 
instrumentation  training. 


TABLE  4.  MODULE  I  -  LIMITED  EXPERIENCE,  PRETRAINING  VERSUS  POSTTRAINING  COMPARISON 


3.2  EMERGENCY  SHIPHANDLING  — 

MODULE  II 

The  results  pertaining  to  Emergency 
Shiphandling  (Module  II)  are  presented 
below  with  regard  to  Training  Effec¬ 
tiveness  and  Pilot  Experience. 


Training  Effectiveness 

The  data  pertaining  to  the  training 
effectiveness  achieved  during  the 
emergency  shiphandling  module  are  pre¬ 
sented  in  Tables  6  through  8.  The 
primary,  secondary  and  four  third- 
order  performance  measures  were  cal¬ 
culated  to  evaluate  training  effec¬ 
tiveness.  The  difference  in  the  fre¬ 
quency  of  channel  excursions  between 
the  pretest  and  the  posttest  was  the 
primary  measure  of  training  effective¬ 
ness.  The  frequency  of  channel  excur¬ 
sions  was  significantly  reduced  from 
six  excursions  to  zero  excursions  (p 
0.02)  as  a  result  of  this  training. 
This  demonstrates  a  significant 
bottom-line  gain  in  pilot  performance 
as  a  result  of  the  training  program. 
The  minimum  distance  to  the  channel 
boundary,  the  secondary  performance 
measure,  was  found  to  be  marginally 
significant,  but  also  showing  a  change 
in  pilot  performance  representative  of 
positive  training  effectiveness.  The 
minimum  distance  to  the  channel 
boundary  increased  from  75.0  feet  to 
129.2  feet  (p^O.10).  It  should  be 
noted,  the  minimum  distance  to  the 
channel  boundary  as  used  in  this  in¬ 
vestigation  is  a  very  conservative 
estimate  of  positive  changes  in  per¬ 
formance  due  to  the  manner  in  which  it 
was  calculated  (see  paragraph  2.10). 


Three  of  the  four  third-order  perfor¬ 
mance  measures  showed  a  significant 
improvement  as  a  result  of  the  emer¬ 
gency  shiphandling  training.  The  mean 
number  of  rudder  orders  prior  to  a  30 


degree  heading  change  during  the  turn¬ 
ing  around  maneuver  was  reduced  from 
7.0  to  4.4  (p^0.05);  the  mean  number 
of  course  orders  prior  to  the  30 
degree  heading  change  was  increased 
from  1.9  to  4.3  (p^0.05);  and  the 
mean  number  of  engine  orders  prior  to 
the  30  degree  heading  change  was  re¬ 
duced  from  6.2  to  3.2  (p^0.02). 
These  three  performance  measures  ad¬ 
dress  the  behavior  of  the  pilots  dur¬ 
ing  the  critical  early  stages  of  the 
turning  around  maneuver  (i.e.,  the 
greater  part  of  the  turning  around 
maneuver  during  which  speed  is  taken 
off,  and  the  ownship  heading  is  sub¬ 
stantially  changed).  The  number  of 
rudder  orders  and  engine  orders  prior 
to  the  30  degree  heading  change  were 
found  to  be  significantly  reduced, 
while  the  number  of  course  orders  was 
found  to  significantly  increase  during 
this  period.  The  technique  addressed 
in  Module  II  for  turning  around  empha¬ 
sizes  incremental  changes/modifica¬ 
tions  to  ownship  course  in  comparison 
with  the  channel  centerline  course, 
thus  promoting  the  changes  that  were 
observed.  These  findings  further  sub¬ 
stantiate  the  effectiveness  of  simula¬ 
tor-based  emergency  shiphandling 
training  for  pilots. 

Pilot  Experience 

The  training  effectiveness  data  for 
the  groups  of  pilots  with  limited  and 
extensive  experience  are  summarized  in 
Tables  7  and  8. 

Limited  Experience.  The  only 
performance  measure  to  reflect  a  sig¬ 
nificant  change  for  pilots  with 
limited  experience  was  the  minimum 
distance  to  the  channel  boundary, 
which  increased  from  60.0  feet  to 
170.0  feet  (p^O.05).  This  significant 
finding  suggests  that  the  module  was 
effective  for  training  this  particular 
group  of  pilots.  The  number  of  chan¬ 
nel  excursions,  the  primary  perfor- 


Non-significant 

Not  appropriate  due  to  unequal  variances 


LIMITED  EXPERIENCE  PRETRAINING  VERSUS  POSTTRAINING  COMPARISON 
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mance  measure,  did  not  show  a  signifi¬ 
cant  difference  between  the  pretest 
and  posttests,  even  though  the  fre¬ 
quency  of  excursions  was  reduced  from 
three  to  zero  as  a  result  of  the  emer¬ 
gency  shiphandling  training.  This 
lack  of  statistical  significance  is 
likely  due  to  the  small  number  of  data 
points  in  this  sample  (i.e.,  five). 
This  interpretation  appears  particu¬ 
larly  appropriate  in  view  of  the  sig¬ 
nificant  results  on  the  conservative 
measure  of  minimum  distance  to  the 
channel  boundary.  This  minimum  dis¬ 
tance  was  increased  nearly  three  times 
between  the  pretest  and  posttest,  de¬ 
monstrating  a  substantial  change  in 
performance.  None  of  the  third-order 
performance  measures  were  found  to  be 
significant  for  this  group,  even 
though  their  means  all  changed  in  the 
expected  directions  as  a  result  of  the 
training  program. 

Extensive  Experience.  Neither  the 
primary  or  secondary  performance  mea¬ 
sures  were  found  to  show  a  significant 
change  as  a  result  of  Module  II  for 


pilots  with  extensive  experience 
(Table  8).  It  should  be  noted,  how¬ 
ever,  that  the  number  of  channel  ex¬ 
cursions  was  reduced  from  three  on  the 
pretest  to  none  on  the  posttest.  This 
is  similar  to  that  found  for  the 
pilots  with  limited  experience.  The 
fact  that  this  measure  was  significant 
overall,  for  both  groups  of  pilots 
combined  as  discussed  above,  strongly 
suggests  that  the  lack  of  significance 
for  the  nearly  identical  subgroups  is 
due  to  sample  size  only. 

Two  of  the  four  third-order  perfor¬ 
mance  measures  were  found  to  yield  a 
significant  change  in  performance. 
The  number  of  rudder  orders  prior  to 
the  30  degree  heading  change  was  re¬ 
duced  from  8.6  to  4.7  (p±^0.05).  The 
number  of  engine  orders  during  this 
period  was  reduced  from  6.0  to  2.7 
(p^O.lO).  The  significance  of  these 
results  suggests  that  pilot  behavior 
did  change  as  a  result  of  the  training 
program,  although  this  was  not  demon¬ 
strated  in  the  bottom-line  primary  and 
secondary  performance  measures. 


CHAPTER  4 


DISCUSSION 


The  results  obtained  during  this  in¬ 
vestigation  yielded  a  number  of  signi¬ 
ficant  changes  in  pilot  performance 
and  behavior  as  a  result  of  the  train¬ 
ing  program.  These  data  may  be  inter¬ 
preted  in  a  number  of  ways.  This  sec¬ 
tion  interprets  the  results  in  terms 
of  the  major  issues  under  investiga¬ 
tion,  and  reaches  summary  conclu¬ 
sions.  The  discussion  is  divided  into 
four  major  subsections,  the  first 
three  of  which  address  the  simulator 
data,  and  the  fourth  which  addresses 
an  analysis  of  the  pilot  opinion  ques¬ 
tionnaires.  The  subsections  are:  (1) 
effectiveness  of  simulator-based 
training  for  pilots,  (2)  pilot  skill 
areas  for  which  simulator-based  train¬ 
ing  would  be  most  effective,  (3)  the 
experience  level  of  pilots  most  likely 
to  benefit  from  simulator-based  train¬ 
ing,  and  (4)  pilot  opinions  regarding 
the  potential  of  simulator-based 
training. 

4.1  TRAINING  EFFECTIVENESS 

The  effectiveness  of  simulator-based 
training  for  pilots  is  the  issue  of 
primary  concern  to  this  investiga¬ 
tion.  The  substantial  number  of  per¬ 
formance  measures  that  show  a  signifi¬ 
cant  increase  in  pilot  performance  as 
a  result  of  training  for  both  modules 
strongly  demonstrates  the  potential 
effectiveness  of  simulator-based 
training  for  pilots.  Nine  out  of  12 
performance  measures  across  both 
modules  showed  significant  gains  in 
performance;  the  primary  performance 
measures  in  both  modules  yielded  a 
significant  positive  training  gain. 


Hence,  it  can  be  concluded  that  pilots 
may  potentially  benefit  from  simula¬ 
tor-based  training. 

This  finding  is  extremely  important. 
It  should  be  cautioned,  however,  that 
this  finding  does  not  imply  that  all 
pilots  can  benefit  from  any  form  of 
simulator-based  training.  Rather,  it 
shows  the  potential  benefits  that  can 
accrue  from  appropriate  training  pro¬ 
grams.  The  areas  in  which  simulator- 
based  training  may  be  useful  for 
pilots  is  discussed  further  below. 

4.2  AREAS  OF  PILOT  TRAINING 

The  analysis  preceding  the  simulator 
training  experiment  identified  several 
areas  of  pilot  skill  for  which  simula¬ 
tor-based  training  might  be  effective 
(see  Chapter  2,  Methodology).  The 
Advanced  Instrumentation  and  Emergency 
Shiphandling  areas  were  selected  from 
the  candidate  set  on  the  basis  of 
their  potential  training  effectiveness 
within  the  constraints  imposed  on  this 
experiment.  The  overall  gain  in  per¬ 
formance  as  a  result  of  both  Module  I 
and  Module  II  training  programs,  as 
discussed  above  regarding  training 
effectiveness,  demonstrates  the  poten¬ 
tial  of  simulator-based  training  for 
both  Advanced  Instrumentation  and 
Emergency  Shiphandling.  Four  of  the 
six  performance  measures  for  the  Ad¬ 
vanced  Instrumentation  module  and  five 
of  the  six  performance  measures  for 
the  Emergency  Shiphandling  module  were 
found  to  show  significant  performance 
gains.  Furthermore,  not  only  were  the 
gains  significant,  but  the  magnitude 


of  differences  were  substantial  for 
most  measures.  Hence,  it  can  be  con¬ 
cluded  that  both  Advanced  Instrumenta¬ 
tion  and  Emergency  Shiphandling  areas 
can  be  effectively  addressed  for 
pilots  via  simulator-based  training 


an  area  of  skill  in  which  pilots  could 
potentially  gain  from  simulator-based 
training.  The  primary  performance 
measure  associated  with  maintaining  a 
1.0  nm  radius  is  the  summation  of 
position  errors  from  that  ideal  track 


programs. 

The  finding  that  effective  training 
for  pilots  was  achieved  on  each  of  the 
two  training  modules  selected  suggests 
that  the  highly  structured  investiga¬ 
tive  approach  used  for  selection  has  a 
high  degree  of  validity.  That  is,  the 
two  areas  chosen  for  this  experiment 
were  selected  from  a  larger  group 
similarly  identified,  and  thus  to  some 
extent  are  representative  of  the 
rationale  that  was  used  to  select  each 
of  the  potential  areas.  The  strong 
positive  findings  in  this  experiment 
tend  to  verify  the  structured  analysis 
process  used,  and  thus  further  suggest 
that  the  other  training  areas  similar¬ 
ly  identified  may  also  have  good  po¬ 
tential  for  training  benefit  from  a 


through  the  turn;  as  discussed  in  the 
Methodology  section,  it  is  calculated 
on  the  basis  of  four  data  lines  spaced 
throughout  the  turn.  The  ownship 
position  throughout  the  constant  ra¬ 
dius  turn  was  much  more  closely 
aligned  with  the  ideal  1.0  nm  radius 
of  that  turn  as  a  result  of  the  train¬ 
ing  program  (Figure  5).  The  mean  dis¬ 
crepancy,  in  terms  of  the  summed  de¬ 
viations  at  the  four  data  lines,  was 
nearly  halved  as  a  result  of  the 
training  program  (734  feet  versus  384 
feet).  It  provides  strong  evidence 
for  the  effectiveness  of  simulator- 
based  training  in  addressing  advanced 
instrumentation  aids  for  piloting. 
This  increase  in  trackkeeping  perfor¬ 
mance  provided  improved  vessel  naviga¬ 
tion  in  terms  of  fixing  ownship's 


simulator-based  training  approach. 

A  further  discussion  of  the  findings 
pertaining  to  each  module  of  the 
training  program  is  presented  below. 

Advanced  Instrumentation 

The  benefits  accrued  as  a  result  of 
the  Advanced  Instrumentation  training 
module  are  shown  in  Figures  5  through 
9.  The  performance  objective  of  this 
module  was  to  maintain  a  1.0  nm  radius 
through  the  turn,  using  available  in¬ 
strumentation  (e.g.,  rate-of-turn  in¬ 
dicator,  doppler  speed  log,  racon). 
Being  able  to  maintain  a  perfect  1.0 
nm  radius  using  these  instruments 
would  provide  the  pilot  with  addition¬ 
al  relevant  information  under  condi¬ 
tions  of  reduced  visibility  (i.e., 
ownship  position  through  the  turn). 
Since  these  instruments  are  not  com¬ 
monly  available  on  most  ships  or  in 
most  pilotage  waters,  they  represented 


position.  It  was  achieved  as  a  result 
of  maintaining  the  vessel  rate-of-turn 
more  closely  to  the  ideal  (0.11  de¬ 
grees  per  minute)  for  this  turn;  a 
corresponding  improvement  in  the  mean 
sum  of  differences  between  the  actual 
rate-of-turn  and  the  desired  was 
achieved  (Figure  6). 

A  direct  benefit  of  maintaining  a  con¬ 
stant  radius  through  the  turn  is  that 
the  distance  to  the  channel  boundary 
is  maximized,  placing  ownship  in  an 
optimum  position  throughout  the  turn 
and  minimizing  the  chance  of  an  excur¬ 
sion  beyond  the  channel  boundary.  The 
minimum  distance  to  the  channel 
boundary,  which  is  a  single  point  per¬ 
formance  measure,  should  yield  perfor¬ 
mance  similar  to  the  sum  of  distances 
off  the  1.0  nm  radius.  That  is,  a 
larger  minimum  distance  to  the  channel 
boundary  should  be  obtained  when  the 
constant  radius  turn  (i.e.,  in  this 
case  1.0  nm)  is  best  maintained.  As 
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Figure  5.  Module  I  Training  Effectiveness 
Sum  of  Distance  Off  1  nm  Radius 
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Figure  6.  Module  I  Training  Effectiveness  — 

Mean  Sum  of  Differences  Between  Actual  and  Desired  Rate-of-Turn 
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Figure  7.  Module  I  Training  Effectiveness  — 
Minimum  Distance  to  Channel  Boundary 
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Figure  8.  Module  I  Training  Effectiveness  — 
Number  of  Course  Orders 
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Figure  9.  Module  I  Training  Effectiveness  —  Number  of  Rudder  Orders 


can  be  seen  in  Figure  7,  the  minimum 
distance  to  the  channel  boundary  in¬ 
creased  substantially  as  a  result  of 
training,  similar  to  that  which  oc¬ 
curred  for  the  sum  of  distances  off 
the  1.0  nm  radius.  This  finding 
further  substantiates  the  potential  of 
simulator-based  training  of  pilots 
with  regard  to  advanced  instrumenta¬ 
tion. 


The  technique  used  to  achieve  a  con¬ 
stant  rate-of-turn  maneuver  is  sensi¬ 
tive  to  the  initial  ownship  position 
when  entering  the  turn,  ownship  speed, 
and  maintenance  of  an  optimum  rate-of- 
turn  configured  for  that  particular 
turn.  Course  headings,  typically,  are 
not  relevant  in  conducting  this  type 
of  maneuver  since  the  rate  is  sensi¬ 
tive  to  amount  of  rudder.  Hence, 
course  orders  would  be  expected  to 
decrease,  and  rudder  orders  would  be 
expected  to  correspondingly  increase. 
The  results,  although  not  conclusive 
in  this  regard,  strongly  suggest  that 
these  changes  in  behavior  did  occur. 
The  mean  number  of  course  orders  used 
to  achieve  the  turn  prior  to  the  ad¬ 
vanced  instrumentation  training  was 
2.25  (Figure  8).  After  the  training 


program  was  completed,  the  mean  number 
of  course  orders  used  was  reduced  to 
1.17.  This  halving  of  the  number  of 
course  orders  as  a  result  of  the 
training  program  was  marginally  signi¬ 
ficant  (p^O.10).  This  level  of  sig¬ 
nificance  has  been  accepted  in  the 
past  for  this  type  of  research,  due  to 
the  relatively  low  power  of  the  sta¬ 
tistical  tests  resulting  from  the 
highly  variable  performance  data 
usually  obtained  in  this  type  of 
experiment.  Variable  in  simulator 
training  research  data  can  result  from 
several  factors.  However,  it  should 
be  noted  that  a  primary  factor  is 
usually  the  relatively  small  number  of 
subjects  used  in  order  to  minimize 
cost  of  the  experiment.  Based  on  the 
significance  of  the  primary  and 
secondary  measures,  therefore,  the 
difference  in  course  orders  as  a  re¬ 
sult  of  the  training  program  is  likely 
to  be  significant. 


Conversely,  the  number  of  rudder 
orders  was  not  found  to  differ  signi¬ 
ficantly  as  a  result  of  the  training 
program,  although  the  mean  value  did 
increase  slightly  (Figure  9).  Never¬ 
theless,  the  proportionate  number  of 
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rudder  orders  to  course  orders  appears 
to  have  changed  as  expected,  suggest¬ 
ing  that  the  pilots  modified  their 
behavior  in  transiting  the  constant 
radius  turn  as  a  result  of  the  simula¬ 
tor-based  training  program. 

In  summary,  the  Advanced  Instrumenta¬ 
tion  training  program  resulted  in 
changes  in  pilot  behavior  and  perfor¬ 
mance  as  expected,  achieving  improve¬ 
ment  in  shiphandling  performance  per¬ 
taining  tc  constant  radius  turn  tech¬ 
niques.  he  empirical  findings  show 
that  significant  benefits  for  pilots 
may  exist  as  a  result  of  participation 
in  simulator -based  Advanced  Instrumen¬ 
tation  training. 

Emergency  Shiphandling 

The  simulator-based  training  program 
was  found  to  be  effective  for  the  type 
of  emergency  shiphandling  addressed 
during  this  program,  as  evidenced  by 
the  significant  changes  in  performance 
summarized  in  Table  6.  The  bottom- 
line  measure  of  shiphandling  perfor¬ 
mance  is  the  number  of  collisions, 
rammings,  and  groundings,  and  in  terms 
of  this  experiment,  the  frequency  of 
channel  excursions.  The  turning 
around  maneuver  that  was  addressed  in 
this  module  was  believed  to  be  quite 
difficult,  to  the  extent  that  a  signi¬ 
ficant  potential  for  going  beyond  the 
channel  boundaries  existed.  Hence, 
this  became  the  primary  performance 
measure.  The  significant  difference 
in  the  number  of  channel  excursions 
before  and  after  the  training  (i.e., 
six  excursions  before,  none  after¬ 
wards)  alone  attests  to  the  potential 
effectiveness  of  simulator-based 
training  for  this  area  of  pilot 
skills.  The  number  of  other  signifi¬ 
cant  performance  measures  also  showing 
a  training  gain  (i.e.,  3  of  the  re¬ 
maining  5)  further  substantiates  this 
finding.  A  summary  of  the  findings 
pertaining  to  emergency  shiphandling 


are  presented  in  Figures  10  through  13 
and  discussed  below. 

The  six  channel  excursions  resulting 
from  the  pretests  shows  the  difficulty 
of  turning  a  large  vessel  around  in  a 
relatively  narrow  channel.  The 
absence  of  any  channel  excursions  on 
the  posttest  demonstrates  the  effec¬ 
tiveness  of  simulator-based  training 
in  polishing  the  skills  of  experienced 
pilots.  The  turning  around  maneuver 
was  accomplished  by  using  a  specific 
technique,  which  has  been  developed  by 
Dutch  pilots  in  the  port  of  Rotter¬ 
dam.  The  effectiveness  of  this  rela¬ 
tively  brief  (only  8-hours  long) 
training  program  atests  to  the  poten¬ 
tial  effectiveness  of  simulator-based 
training  for  pilots,  as  well  as  the 
high-level  of  skills  and  knowledge 
possessed  by  the  pilots  participating 
in  this  experimental  program.  These 
pilots  were  able  to  rapidly  learn  and 
apply  the  Dutch  technique  in  a  diffi¬ 
cult  shiphandling  situation.  Commer¬ 
cial  training  programs  would  certainly 
spend  considerably  more  time  address¬ 
ing  and  practicing  appropriate  emer¬ 
gency  shiphandling,  as  do  the  Dutch 
pilots  during  their  week  long  re¬ 
fresher  training  program  at  Wageningen. 

The  minimum  distance  to  the  channel 
boundary  during  the  turn  around  maneu¬ 
ver  was  found  to  improve  in  direct 
correspondence  to  the  number  of  chan¬ 
nel  excursions,  increasing  from  75 
feet  to  129  feet  (Figure  10).  This 
represents  an  increase  of  72%  as  a 
result  of  the  training  program.  Al¬ 
though  the  level  of  statistical  signi¬ 
ficance  of  this  increase  is  marginal 
(p^O.10),  the  method  of  calculation 
together  with  the  number  of  channel 
excursions  suggest  that  the  obtained 
level  of  statistical  significance  is  a 
conservative  estimate.  The  computed 
mean  distances  were  based  on  the  actu¬ 
al  distance  for  each  vessel  that  did 
not  go  beyond  the  channel  boundaries, 
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Figure  12.  Module  II  Training  Effectiveness  — 
Number  of  Course  Orders 
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Figure  13.  Module  II  Training  Effectiveness  — 
Number  of  Engine  Orders 


and  a  value  of  "zero"  for  those  ves¬ 
sels  that  went  beyond  the  channel 
boundary.  Thus,  these  mathematical 
computations  included  the  ratios  of 
distances  for  vessels  staying  within 
the  channel,  while  adding  only  a  zero 
(i.e.,  rather  than  a  negative  value) 
for  those  that  went  outside  the  chan¬ 
nel.  The  resulting  performance  mea¬ 
sure,  therefore,  is  weighted  in  favor 
of  providing  a  within-channel  value. 
Hence,  the  difference  between  pretest 
and  posttest  performance  as  calculated 
for  this  measure  would  be  conserva¬ 
tive.  This  marginally  significant 
result  should  thus  be  interpreted  as 
definitely  significant.  The  simulator 
was  found  to  be  an  effective  medium 
for  training  emergency  shiphandling 
techniques  to  pilots. 

The  turning  around  technique  developed 
by  the  Dutch  places  considerable  em¬ 
phasis  on  the  early  stages  of  the 
turning  around  process.  During  the 
early  period  the  speed  of  the  vessel 
should  be  reduced  and  the  vessel's 
heading  changed  appropriately  in  cor¬ 
respondence  with  the  procedure  out¬ 
lined  in  Table  2.  A  30  degree  heading 
change  with  respect  to  the  channel 
centerline  was  chosen  in  this  experi¬ 
ment  to  represent  the  point  at  which 
the  early  stages  of  the  turn  around 
maneuver  were  concluded.  This  was 
used  as  a  reference  datum  for  evaluat¬ 
ing  performance.  The  turning  around 
technique  relies  on  a  difference  be¬ 
tween  the  shipheading  and  the  channel 
centerline's  course  in  correspondence 
with  the  current's  magnitude  and  di¬ 
rection,  and  ownship  speed.  Hence,  it 
was  expected  that  the  number  of  rudder 
orders  given  would  decrease,  and  the 
number  of  course  orders  would  con¬ 
versely  increase,  as  a  result  of  the 
training  program.  The  number  of  rud¬ 
der  orders  issued  prior  to  a  30  degree 
heading  change  was  significantly  re¬ 
duced  as  a  result  of  the  training  pro¬ 
gram,  from  7.0  on  the  pretest  to  4.4 


on  the  posttest  (Figure  11).  Corre¬ 
spondingly,  the  number  of  course 
orders  were  significantly  increased 
from  2.2  on  the  pretest  to  4.1  on  the 
posttest  (Figure  12).  These  findings 
are  consistent  with  the  Dutch  tech¬ 
nique  employed,  and  as  expected.  They 
provide  the  rationale  for  improved 
pilot  performance  in  turning  the  large 
vessel  around  in  a  narrow  channel,  in 
terms  of  changes  in  pilot  shiphandling 
behavior  during  that  maneuver. 

The  early  stages  of  the  Dutch  turning 
around  technique  focus  on  balancing 
the  ship's  cross  channel  velocity  vec¬ 
tor  with  that  of  the  current,  and 
hence  require  the  pilot  to  modify  the 
ship's  heading  as  a  result  of  changes 
in  ownships  speed.  As  the  speed  de¬ 
creases,  greater  heading  changes  are 
made  to  counter  balance  the  current, 
and  thus  to  ideally  keep  the  center  of 
ownship  in  the  center  of  the  channel 
(note,  course  would  ideally  remain 
along  the  channel  centerline).  If 
sufficient  distance  ahead  is  avail¬ 
able,  little  if  any  use  of  the  engine 
during  these  early  stages  is  neces¬ 
sary.  Rather,  the  engine  is  stopped, 
and  as  the  ship's  speed  falls  off  over 
time  course  corrections  are  made  to 
counter  balance  the  current.  It  was 
expected,  therefore,  that  the  training 
program  would  result  in  fewer  engine 
orders  during  the  early  stages  of  the 
turn  around  maneuver.  A  significant 
reduction  in  the  number  of  engine 
orders  given  was,  in  fact,  obtained 
(Figure  13).  The  mean  number  of 
engine  orders  dropped  from  2.25  prior 
to  the  training  program  to  1.17  fol¬ 
lowing  the  training  program.  This 
finding  further  describes  the  changes 
in  pilot  behavior  that  led  to  improved 
turning  around  performance. 

In  summary,  the  Emergency  Shiphandling 
training  module  was  found  to  be  effec¬ 
tive.  Significantly  fewer  channel 
excursions  occurred  after  the  training 


program,  with  the  pilots  using  corre¬ 
spondingly  less  of  the  channel  width 
to  turn  the  vessel  around.  These 
emergency  shiphandling  performance 
improvements  were  directly  related  to 
observed  changes  in  pilot  behavior  as 
a  result  of  the  training  program. 

4.3  LEVEL  OF  EXPERIENCE 

The  third  major  issued  addressed  by 
this  experiment  is  the  difference  in 
the  effectiveness  of  simulator-based 
training  for  different  groups  of 
pilots.  The  experiment  investigated 
pilots  with  limited  experience  (i.e., 
less  than  5-years)  and  pilots  with 
extensive  experience  (i.e.,  greater 
than  10-years).  These  two  categories 
of  pilot  experience,  although  appear¬ 
ing  straightforward,  are  actually 
somewhat  complex.  The  differences  in 
entry  qualifications  and  training  pro¬ 
grams  for  the  various  pilot  organiza¬ 
tions  make  it  difficult  to  categorize 
the  amount  of  experience  a  pilot  has. 
For  example,  some  pilots  enter  the 
piloting  profession  after  having  had  a 
considerable  amount  of  experience  as  a 
deck  officer  (i.e.,  on  a  sea-going 
vessel,  a  tug  boat,  etc.).  Although 
these  individuals  may  have  had  only 
limited  experience  as  a  pilot  since 
joining  the  pilot  organization,  they 
may  have  had  considerable  experience 
as  deck  officers  and  in  working  with 
pilots.  Other  pilot  organizations,  at 
the  opposite  extreme,  do  not  have  any 
prerequisite  experience  requirements 
for  entering  the  piloting  profession. 
These  organizations,  however,  typical¬ 
ly  have  a  very  extensive  training  pro¬ 
gram  spanning  a  number  of  years.  The 
pilot/subjects  within  both  the  limited 
and  extensive  experienced  groups  re¬ 
presented  both  types  of  backgrounds, 
although  they  all  met  the  established 
criteria  for  years  experience  as  a 
licensed  pilot  within  their  assigned 
groups.  As  a  result,  these  classifi¬ 


cations  should  be  viewed  as  approxi¬ 
mate. 

The  analysis  investigated  each  of  the 
two  pilot  experience  groups  indepen¬ 
dently,  with  regard  to  each  of  the  two 
training  modules.  Particular  caution 
should  be  exercised  in  interpreting 
the  findings  relating  to  experience, 
since  relatively  small  sample  sizes 
were  used  in  each  group.  The  results 
are  discussed  below  regarding  Advanced 
Instrumentation  and  Emergency  Ship¬ 
handling. 

Advanced  Instrumentation 

The  pilots  with  extensive  experience 
appear  to  have  benefited  more  from  the 
Advance  Instrumentation  training  than 
did  those  with  limited  experience. 
The  differences  in  training  effective¬ 
ness  for  the  two  groups  are  reflected 
in  the  three  performance  measures  pre¬ 
sented  in  Figures  14  through  19.  The 
track  radius  error  showed  a  signifi¬ 
cant  positive  training  gain  for  the 
extensive  experience  group,  while  a 
similar  result  was  not  obtained  for 
the  limited  experience  group  (Figure 
14).  This  finding  demonstrates  than 
the  Advanced  Instrumentation  training 
program  was  effective  for  the  experi¬ 
enced  group  of  pilots.  The  lack  of  a 
significant  finding  on  this  measure 
for  the  limited  experience  group  does 
not  show  that  the  training  was  inef¬ 
fective;  rather,  it  indicates  that  the 
data  were  insufficient  to  demonstrate 
an  effective  gain.  It  should  be  noted 
that  the  limited  experience  group  did 
achieve  a  relatively  large  difference 
in  performance  (i.e.,  626.5  feet  on 
the  pretest  versus  447.2  feet  on  the 
posttest),  even  though  it  was  not 
found  to  be  statistically  signifi¬ 
cant.  This  lack  of  statistical  signi¬ 
ficance  may  be  due  to  the  power  of  the 
test,  resulting  from  the  relatively 
small  sample  size.  A  relatively  small 
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Figure  14.  Level  of  Experience,  Module  I 
Sum  of  Differences  off  1.0  nm  Radius 
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Figure  15.  Level  of  Experience,  Module  I  — 

Sum  of  Differences  Between  Actual  and  Desired  Rate-of-Turn 
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Figure  16.  Level  of  Experience,  Module  I  — 
Minimum  Distance  to  Channel  Boundary 
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Figure  17.  Level  of  Experience,  Module  II  — 
Minimum  Distance  to  Channel  Boundary 
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Figure  18.  Level  of  Experience,  Module  II  -- 
Number  of  Rudder  Orders 
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Figure  19.  Level  of  Experience,  Module  II  -- 
Number  of  Engine  Orders 


number  of  pilot/subjects  were  used  in  groups  appear  somewhat  opposite  to 

this  condition  due  to  cost  and  experi-  those  found  for  the  advanced  instru¬ 
mental  design  considerations.  This  mentation  module.  The  pilots  with 

results  in  relatively  low  statistical  limited  experience  appear  to  have 

power  for  these  types  of  real-world  benefited  greater  from  this  module 

experiments,  and  greatly  increases  the  than  did  those  with  extensive  experi- 

chances  of  not  obtaining  statistically  ence.  The  primary  performance  mea- 

significant  results  even  though  they  sure,  frequency  of  channel  excursions, 

do  exist.  did  not  show  a  significant  difference 

for  either  group  separately  as  a  re- 
Both  groups  of  pilots  achieved  a  sig-  suit  of  the  training  program.  Since 

nificant  positive  training  gain  in  the  this  measure  was  found  to  be  statisti- 

rate-of-turn  error,  a  secondary  per-  cally  significant  with  both  groups 

formance  measure  (Figure  15).  These  combined,  it  can  therefore  be  con- 

results  support  the  above  contention  eluded  that  the  lack  of  significance 

that  the  training  program  was  effec-  for  either  group  when  separated  is  due 

tive  for  both  groups,  and  that  the  to  the  small  number  of  pilots  in  each 

lack  of  significance  of  the  substan-  group  and  the  resultant  low  statisti¬ 
cal  track  radius  error  reduction  was  cal  power, 

due  to  the  power  of  the  statistical 

analysis.  The  limited  experience  pilots  signifi¬ 

cantly  improved  their  minimum  distance 
The  minimum  distance  to  the  channel  to  the  channel  boundary  as  a  result  of 

boundary  was  found  to  be  significant  the  training  program  (Figure  17). 

for  the  extensive  experience  group,  This  measure  is  closely  related  to  the 

but  not  for  the  limited  experience  number  of  channel  excursions.  This 

group  (Figure  16).  The  mean  minimum  group  of  pilots  on  the  posttest  nearly 

distance  to  the  channel  boundary  did  tripled  their  pretest  minimum  distance 

not  change  appreciably  for  the  limited  to  the  channel  boundary  (i.e.,  60  feet 

experienced  group,  hence  the  lack  of  to  170  feet).  The  pilots  with  exten- 

statistical  significance.  This  find-  sive  experience,  on  the  ether  hand, 

ing  supports  the  above  contention  that  achieved  a  relatively  small  increase 

the  extensive  experience  group  bene-  in  the  minimum  distance  to  the  channel 

fited  more  from  the  Advanced  Instru-  boundary  (86  feet  to  100  feet),  which 

mentation  module  than  did  the  limited  was  not  statistically  significant.  It 

experience  group.  is  concluded  on  the  basis  of  these 

findings  that  the  pilots  with  limited 
In  summary,  the  experimental  results  experience  benefited  more  from  the 

indicate  that  both  groups  of  pilots  Emergency  Shiphandling  module  than  did 

benefited  from  the  Advanced  Instrumen-  the  pilots  with  extensive  experience, 

tation  training  module.  The  extensive 

experience  group,  however,  was  found  The  extensive  experience  pilots  did, 

to  benefit  more  from  this  training  interestingly,  change  their  behavior 

then  the  limited  experience  group.  as  a  result  of  this  Emergency  Ship- 

The  reasons  for  this  difference  are  handling  module,  although  it  was  not 

not  apparent  from  the  data  collected.  reflected  in  a  statistically  signifi¬ 

cant  change  in  vessel  performance. 
Emergency  Shiphandling  The  number  of  rudder  orders  and  the 

number  of  engine  orders  used  by  this 
The  training  benefits  of  the  Emergency  group  was  significantly  reduced  as  a 

Shiphandling  module  for  the  two  pilot  result  of  the  emergency  shiphandling 


training  (Figures  18  and  19).  These 
findings  are  in  direct  correspondence 
with  those  for  the  total  group  of 
pilots,  as  discussed  previously;  and 
they  are  in  the  direction  expected. 
These  findings  suggest  that  the  tech¬ 
nique  used  to  achieve  the  vessel  turn 
around  by  the  extensive  experience 
group  of  pilots  changed  as  a  result  of 
the  training  program.  Their  level  of 
activity  reduced  considerably  in  terms 
of  both  rudder  orders  and  engine 
orders.  Furthermore,  it  should  be 
noted  that  although  a  significant  dif¬ 
ference  in  the  distance  to  the  channel 
boundary  was  not  achieved,  the  mean 
difference  that  did  occur  was  in  the 
positive,  rather  than  negative,  train¬ 
ing  gain  direction.  Hence,  as  a  re¬ 
sult  of  these  significant  changes  in 
behavior,  it  is  concluded  that  the 
pilots  with  extensive  experience  also 
showed  training  gain  as  a  result  of 
this  module. 

In  summary,  the  above  findings  suggest 
differences  in  the  potential  benefits 
to  be  accrued  from  simulator-based 
training  on  the  basis  of  the  pilot/ 
student  level  of  experience.  Although 
both  groups  of  pilots  showed  benefit 
from  each  of  the  two  training  modules, 
those  with  extensive  experience  ap¬ 
peared  to  gain  more  from  the  Advanced 
Instrumentation  module,  while  those 
with  limited  experience  appeared  to 
gain  more  from  the  Emergency  Ship¬ 
handling.  The  findings  are  not  strong 
with  regard  to  the  differences  between 
the  groups,  probably  a  result  of  the 
small  sample  size  used.  Furthermore, 
the  pilots  in  both  groups,  although 
technically  fitting  into  the  respec¬ 
tive  categories,  differ  substantially 
in  their  backgrounds.  Hence,  a  con¬ 
clusion  cannot  be  reached  as  to  which 
groups  of  pilots  will  definitely  bene¬ 
fit  more  from  either  the  Advanced  In¬ 
strumentation  or  the  Emergency  Ship¬ 
handling  training.  On  the  other  hand, 
the  number  of  differences  found  be¬ 


tween  the  two  groups  strongly  sug¬ 
gests,  as  was  expected,  that  the  bene¬ 
fits  to  be  derived  from  the  different 
areas  of  simulator -based  training  of 
pilots  will  depend  on  their  back¬ 
grounds,  such  as  level  of  experience. 
This  finding  was  not  unexpected,  since 
the  benefits  to  be  derived  from  any 
training  program  depend  heavily  on  the 
backgrounds  of  the  particular  stu¬ 
dents.  Hence,  it  is  likely  that 
apprentice  pilots,  limited  experienced 
pilots,  and  extensively  experienced 
pilots  will  derive  differential  bene¬ 
fits  from  the  simulator-based  training 
in  the  various  areas  of  piloting 
skill.  The  extent  of  benefits  to  be 
achieved  from  such  training  by  these 
groups,  however,  cannot  be  determined 
on  the  basis  of  these  data. 

4.4  PILOT  OPINIONS 

Pilots  in  the  United  States  have  re¬ 
ceived  relatively  little  exposure  to 
simulator-based  training  programs. 
Pilots  have  participated  in  simulator 
experiments,  but  these  are  substan¬ 
tially  different  than  a  structured 
training  program  designed  specifically 
to  address  the  improvement  of  piloting 
skills.  A  broad  range  of  opinion 
exists  in  the  piloting  community  re¬ 
garding  the  potential  effectiveness  of 
simulator-based  training.  An  impor¬ 
tant  part  of  the  empirical  data  col¬ 
lected  as  a  part  of  this  experiment, 
therefore,  was  the  opinion  of  pilots 
with  regard  to  the  potential  of  simu¬ 
lator-based  training  as  a  result  of 
their  having  participated  in  a  struc¬ 
tured  training  program.  For  that  rea¬ 
son,  the  pilots  participating  in  this 
experiment  were  viewed  not  only  as 
subjects,  but  also  as  evaluators. 
Discussions  concerning  the  various 
aspects  of  the  training  program  were 
carried  out  during  the  free  moments  of 
the  course.  In  addition,  a  debriefing 
questionnaire  was  administered  to  each 
pilot  individually  at  the  conclusion 


of  the  training  program.  The  intent 
was  to  collect  as  much  information  as 
possible  regarding  the  pilots'  view 
points,  insights,  recommendations,  etc. 

A  production  training  facility  would 
normally  modify  various  aspects  of  the 
program  (i.e.,  including  simulator, 
modules,  etc.)  after  each  time  the 
course  is  presented,  on  the  basis  of 
inputs  from  instructors  and  students. 
This,  of  course,  is  the  approach  the 
Rotterdam  pilots  followed  after  first 
instituting  simulator-based  training. 
For  example,  the  Dutch  spent  approxi¬ 
mately  two  years  in  refining  the 
hydrodynamic  characteristics  of  the 
ship  models  they  were  using,  to  make 
them  acceptable  to  the  pilots.  Due  to 
the  experimental  nature  of  the  train¬ 
ing  program  used  in  this  investiga¬ 
tion,  it  remained  the  same  for  the 
three  times  it  was  presented  (i.e., 
thus  maintaining  the  necessary  degree 
of  experimental  constancy  across  the 
three  groups).  The  information  sum¬ 
marized  herein  provides  an  overview  of 
the  pilots'  observations  and  opinions. 

The  questionnaire  analysis  and  discus¬ 
sion  is  presented  below  in  two  parts. 
The  first  part  provides  a  summary  of 
the  questionnaire  data,  with  regard  to 
the  specific  issues  being  addressed. 
The  second  part  focuses  on  the  larger 
issues  of  interest  to  this  investiga¬ 
tion  (i.e.,  potential  of  simulator- 
based  training  for  pilots),  integrat¬ 
ing  the  questions  and  responses  as 
appropriate. 

Questionnaire  Summary 

The  questionnaire  addressed  a  variety 
of  issues  pertinent  to  the  investiga¬ 
tion,  including  (1)  benefits  gained 
from  each  of  the  three  training 
modules;  (2)  the  potential  for  pilot 
training  regarding  each  of  the  three 
modules;  (3)  problems  and  recommended 
improvements  for  each  of  the  three 


modules;  and  (4)  the  general  potential 
of  simulator-based  training  for 
pilots.  Sixteen  of  the  34  questions 
address  the  general  value  and  poten¬ 
tial  of  simulator-based  training  for 
pilots.  These  are  the  issues  of  pri¬ 
mary  concern  to  this  investigation, 
and  are  discussed  in  detail.  The  re¬ 
maining  questions  address  specific 
details  of  the  particular  experimental 
training  program  provided.  These  de¬ 
tails  are  of  substantial  importance 
from  the  standpoint  of  designing  and 
conducting  pilot  training.  A  brief 
overview  of  these  is  provided.  If  the 
training  program  material  developed 
for  this  investigation  were  to  be  used 
to  provide  additional  training  to 
pilots,  these  latter  observations 
would  be  extremely  important  to  the 
improvement  of  that  program.  All  of 
the  information  collected  from  this 
questionnaire  has  been  carefully  ana¬ 
lyzed  with  regard  to  the  development 
of  the  training  system  guidelines  for 
pilots  which  is  a  separate  document 
being  developed  under  this  project. 

The  methodology  for  administration  of 
the  questionnaire  was  presented  pre¬ 
viously  in  the  methodology  section. 
It  should  be  recalled  that  the  ques¬ 
tionnaires  were  filled  out  by  an  ana¬ 
lyst  during  a  debriefing  session  with 
each  individual  pilot  at  the  comple¬ 
tion  of  the  week's  training  program 
and  experiment.  The  views  expressed 
by  the  pilots  in  the  questionnaire 
reflect  not  only  their  experiences  as 
students/evaluators  in  the  training 
program,  but  also  as  evaluators  who 
spent  considerable  time  discussing 
simulator-based  training  issues. 

A  summary  of  the  specific  responses  of 
the  pilots  to  the  16  primary  questions 
addressing  the  value  and  potential  of 
simulator-based  training  is  presented 
in  Table  9.  The  questions  and  re¬ 
sponses  are  grouped  according  to  each 
of  the  three  training  modules  and  a 
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TABLE  9.  PILOT  QUESTIONNAIRE  SUMMARY  DATA  FOR  SELECTION  QUESTIONS* 


Question** 

Responses 

Advanced  Instrumentation 

1.  Module  rating  (1^*10  highest) 

7.58/10  (mean  value) 

5.  Learned  something  valuable? 

11/12  (yes) 

7.  Valuable  for  apprentices? 

11/12  (yes) 

8.  Valuable  for  experienced  pilots,  periodically? 

11/12  (yes) 

Emergency  Shiphandling 

11.  Module  rating  (1*-*10  highest) 

6.58/10  (mean  value) 

12.  Is  this  valuable? 

11/12  (yes) 

14.  Valuable  for  apprentices? 

11/12  (yes) 

15.  Valuable  for  experienced  pilots,  periodically? 

9/12  (yes) 

Decisionmaking 

19.  Module  rating  (1— ►lO  highest) 

6.29/10  (mean  value)  (8 

pilots) 

20.  Learned  something  valuable? 

3/8  (yes) 

22.  Valuable  for  apprentices? 

7/8  (yes) 

23.  Valuable  for  limited  experience  pilots? 

8/8  (yes) 

24.  Valuable  for  experienced  pilots,  periodically? 

3/8  (yes) 

General 

27.  Simulator  potential  for  apprentices? 

11/12  (yes) 

28.  Simulator  potential  for  licensed  pilots? 

11/12  (yes) 

32.  Beneficial  training  in  generic  port? 

11/12  (yes) 

general  category.  All  of  the  pilots 
participated  in  the  Advanced  Instru¬ 
mentation  and  Emergency  Shiphandling 
training  modules.  Eight  of  the  12 
pilots  were  able  to  partially  partici¬ 
pate  in  the  decisionmaking  training 
module,  while  one  group  of  four  pilots 
were  unable  to  participate  in  this 
module  due  to  simulator  problems. 
Hence,  the  questionnaire  data  pertain¬ 
ing  to  this  module  is  not  considered 
to  be  of  the  same  quality  as  that  of 
the  first  two  modules. 

A  substantial  number  of  specific 
issues  were  raised  by  the  pilots 
regarding  many  of  the  questions. 
These  issues  dealt  with  qualifications 
to  their  answers,  generally  regarding 
(1)  conditions  under  which  their  ans¬ 
wers  would  be  given  one  way  or  another 
(i.e.,  yes  versus  no);  and  (2)  policy 
issues  pertaining  to  how  such  training 
would  come  about.  Several  examples  of 
caveats  expressed  by  pilots  are: 

•  The  piloting  problems  and  instru¬ 
ments  addressed  should  be  specific 
to  each  pilot's  port  area. 

•  The  instrumentation  addressed 
should  a  represent  real  break 
through,  rather  than  commonly 
available  instruments. 

•  The  emergency  shiphandling  tech¬ 
niques  addressed  should  represent 
new  techniques,  rather  than  tech¬ 
niques  with  which  the  pilots  have 
experience. 

•  Although  the  content  of  simulator- 
based  training  has  good  potential, 
its  actual  value  would  depend  on 
local  availability  and  cost. 

•  Simulator-based  training  would  not 
be  an  acceptable  substitute  for 
aspects  of  current  pilot  training 
programs;  rather,  its  value  is  as  a 
supplement  to  current  programs  to 


further  enhance  pilot  skills.  The 
use  of  simulator-based  training 
should  be  voluntary,  rather  than 
mandatory. 

The  final  two  points  in  this  list  were 
expressed  by  most  pilots.  They 
strongly  approve  of  their  current 
training  programs,  but  do  feel  that 
value  may  be  had  from  simulator-based 
training  as  discussed  below.  They 
further  feel  that  the  positive  bene¬ 
fits  to  be  potentially  gained  from 
such  training  would  be  negated  if  it 
were  forced  upon  them.  The  pilots 
believe  that  the  need  for  simulator- 
based  training  and  its  potential  value 
is  a  local  port  issue,  depending  on  a 
variety  of  factors  (e.g.,  current 
training  program,  port  characteris¬ 
tics,  issues  to  be  addressed  by  simu¬ 
lator-based  training,  individual  pilot 
backgrounds,  and  so  on).  When  simula¬ 
tor-based  training  is  provided  it 
should  be  tailored  to  the  needs  of  a 
particular  port.  The  final  issue  is 
the  financing  for  such  training.  The 
use  of  simulator-based  training  by 
pilots  will  directly  depend  on  the 
cost  of  such  training,  and  the  avail¬ 
able  financing. 

Training  Program  Value  and  Potential 

The  questions  addressing  the  value  and 
potential  of  simulator-based  training 
were  generally  concerned  with:  (1) 
the  value  of  that  module  as  experi¬ 
enced  by  the  pilots;  (2)  the  pilots' 
perceived  value  of  that  module  for 
apprentice  training;  and  (3)  the 
pilots'  perceived  value  of  that  module 
for  periodic  refresher  training  of 
licensed  pilots.  The  responses  to 
these  issues  are  presented  in  Table  9. 

Advanced  Instrumentation.  The  Ad¬ 
vanced  Instrumentation  module  was 
evaluated  by  the  pilots  as  having 
value  as  conducted,  and  as  having 
potential  for  both  apprentices  and 


experienced  pilots.  The  pilot  ratings 
of  this  course  had  a  mean  value  of 
7.58  out  of  a  possible  10  points. 
This  above  average  rating  reflects  the 
value  of  the  Advanced  Instrumentation 
module  as  presented.  Each  of  the 
training  modules,  as  noted  previously, 
can  and  should  be  improved  as  a  result 
of  each  application.  The  ratings 
given  to  this  and  the  other  two 
modules  reflect  the  initial  presenta¬ 
tion  of  each;  improvements  could  cer¬ 
tainly  be  made  as  a  result  of  the 
pilots  recommendations  and  the  staff 
recommendations  stemming  from  this 
initial  conduct  of  the  training  pro¬ 
gram.  The  high  proportion  (i.e.,  11/ 
12)  indicating  something  of  value  was 
learned  from  this  module  further  sub¬ 
stantiates  the  rating.  Of  greater 
importance  to  this  investigation,  how¬ 
ever,  is  the  potential  for  such  train¬ 
ing.  A  very  high  proportion  of  the 
pilots  stated  that  the  Advanced  In¬ 
strumentation  type  of  training  had 
some  value  for  both  apprentices  and 
experienced  pilots  (i.e.,  11/12  and 

12/12  respectively).  These  opinions 
are  in  good  correspondence  with  the 
performance  data  collected  regarding 
the  training  effectiveness  of  the  Ad¬ 
vanced  Instrumentation  module. 

The  piloting  skills  and  knowledge  ad¬ 
dressed  under  the  Advanced  Instrumen¬ 
tation  module  are  not  in  common  prac¬ 
tice  in  most  ports,  and  hence  were 
believed  to  have  potential  training 
value.  The  findings,  both  performance 
and  opinion  data,  substantiate  that 
Advanced  Instrumentation  training  does 
have  value  and  potential.  It  should 
be  cautioned,  however  as  noted  above, 
that  these  findings  pertain  to  the 
general  potential  of  such  training. 
As  several  of  the  pilots  stated,  the 
relevancy  of  Advanced  Instrumentation 
training,  and  hence  the  benefits  to  be 
had  by  any  particular  group  of  pilots, 
will  depend  on  the  degree  of  experi¬ 
ence  the  pilots  have  had  with  the  par¬ 


ticular  instruments,  the  local  condi¬ 
tions  in  their  particular  ports  (e.g., 
availability  of  racons),  and  so  on. 
These  findings  demonstrate  the  poten¬ 
tial  value  of  Advanced  Instrumentation 
simulator -based  training;  they  do  not 
address  the  specific  value  of  this 
type  of  training  for  any  particular 
port  or  group  of  pilots.  The  specific 
value  would,  of  course,  depend  on  the 
particular  aspects  of  Advanced  Instru¬ 
mentation  addressed,  as  well  as  the 
pilot  and  port  characteristics. 

Emergency  Shiphandling.  The  pilot 
opinions  regarding  the  value  and  po¬ 
tential  of  the  Emergency  Shiphandling 
module  were  similar  to  those  of  the 
Advanced  Instrumentation  module.  The 
mean  rating  given  to  the  module  as 
taught  was  6.58  out  of  a  possible  10 
points  (see  Table  9).  This  above 
average  value  is  supported  by  the  high 
proportion  that  indicated  they  learned 
something  valuable  from  the  emergency 
shiphandling  module  (i.e.,  11  out  of 
12  pilots).  A  high  proportion  of  the 
pilots  felt  that  this  type  of  training 
would  be  of  value  for  apprentices  (IT 
out  of  12),  while  two-thirds  of  (9  out 
of  12)  indicated  that  periodic  Emer¬ 
gency  Shiphandling  training  would  be 
of  value  for  licensed  pilots.  This  is 
the  type  of  training  that  the  Rotter¬ 
dam/Europort  pilots  participate  in 
every  two  years. 

On  the  basis  of  the  questionnaire  re¬ 
sponses  it  can  be  concluded  that 
pilots  believe  simulator-based  Emer¬ 
gency  Shiphandling  training  has  value 
for  both  apprentices  and  experienced 
pilots.  This  conclusion  corresponds 
well  with  the  performance  data  from 
the  training  program,  which  demon¬ 
strated  an  improvement  in  Emergency 
Shiphandling  skills  as  a  result  of  the 
training.  Of  the  three  pilots  that 
felt  the  Emergency  Shiphandling  train¬ 
ing  would  not  be  of  value  to  experi¬ 
enced  pilots,  one  said  it  was  because 


the  techniques  were  not  new  to  him; 
another  stated  it  would  depend  on  the 
situation  addressed.  If  the  appropri¬ 
ate  issues  are  addressed  during  this 
module,  therefore,  the  proportion  of 
pilots  supporting  Emergency  Ship¬ 
handling  for  licensed  pilots  would  be 
similar  to  that  supporting  the  Ad¬ 
vanced  Instrumentation  training. 

Decisionmaking.  The  pilot  opinions 
regarding  the  Decisionmaking  module 
were  found  to  differ  from  those  of  the 
previous  two  modules.  A  difference  of 
opinion  was  expected,  since  it  was 
believed  that  the  Decisionmaking 
module  would  be  most  appropriate  for 
pilots  with  limited  experience,  and 
perhaps  also  for  apprentice  pilots. 
Furthermore,  only  one  group  of  pilots 
completed  the  Decisionmaking  module;  a 
second  group  of  pilots  completed  the 
majority  of  the  Decisionmaking  module, 
while  the  third  group  of  pilots  did 
not  participate  at  all  in  this 
module.  Simulator  problems  caused 
this  discrepant  application  of  the 
third  module.  It  is  likely,  there¬ 
fore,  that  the  pilot  opinions  ex¬ 
pressed  regarding  this  module  are  af¬ 
fected  by  the  manner  in  which  it  was 
presented  during  the  program. 

The  rating  given  to  the  Decisionmaking 
module  by  the  pilots  had  a  mean  value 
of  6.29  out  of  a  possible  10  points. 
Eight  pilots  answered  the  questions 
regarding  this  module.  Although  this 
rating  is  above  five  on  a  scale  of  one 
to  ten,  it  is  also  the  lowest  of  the 
three  modules.  Only  three  of  the 
eight  pilots  felt  that  they  had 
learned  something  valuable  from  this 
module.  On  the  other  hand,  a  high 
proportion  of  the  pilots  felt  that 
this  type  of  training  would  be  of 
value  for  apprentice  pilots  (i.e., 
seven  out  of  eight  pilots),  and  vir¬ 
tually  all  of  the  pilots  felt  it  would 
be  of  value  to  pilots  with  limited 
experience  (i.e.,  eight  of  eight).  On 


the  other  hand,  the  pilots  generally 
felt  that  the  Decisionmaking  module 
would  not  be  of  benefit  as  refresher 
training  for  experienced  pilots  (i.e., 
three  out  of  eight  pilots  felt  it 
would  be  of  benefit).  Hence,  it  can 
be  concluded  on  the  basis  of  the 
pilots  opinions  that  the  Decisionmak¬ 
ing  module  has  potential  value  for  the 
training  of  apprentices  and  pilots 
with  limited  experience.  It  is  not 
believed  to  have  very  much  benefit  for 
the  training  of  experienced  pilots. 
These  findings  cannot  be  further  sub¬ 
stantiated  with  empirically  derived 
data,  since  the  data  were  not  avail¬ 
able  from  the  experiment.  Further¬ 
more,  the  findings  regarding  this 
module  should  be  treated  cautiously 
due  to  the  relatively  small  sample 
size  (i.e.,  eight  pilots)  and  the  dif¬ 
ficulties  encountered  in  conducting 
this  training. 

General  Potential.  Responses  to  the 
three  major  questions  pertaining  to 
the  general  potential  of  simulator- 
based  training  for  pilots  (i.e.,  ques¬ 
tions  27,  28  and  32)  are  included  in 
Table  9.  These  questions  addressed 
the  potential  of  such  training  for 
apprentices  and  licensed  pilots,  and 
also  addressed  the  potential  of  con¬ 
ducting  such  training  in  a  generic 
port  data  base.  Whereas  the  preceding 
questions  addressed  the  specific 
modules  presented  during  the  training 
program  and  the  potential  for  similar 
modules  in  a  production  training  pro¬ 
gram,  these  latter  questions  addressed 
the  broad  issue  of  simulator-based 
training  without  regard  to  any  parti¬ 
cular  modules.  A  large  proportion  of 
the  pilots  (i.e.,  11  out  of  12)  felt 
that  simulator-based  training  had  po¬ 
tential  for  both  apprentices  and  li¬ 
censed  pilots.  As  noted  above,  how¬ 
ever,  this  statement  should  be  inter¬ 
preted  cautiously.  The  pilots  had 
many  specific  concerns  regarding  the 
effective  application  of  simulator- 


based  training,  several  examples  of 
which  have  been  presented  earlier  in 
this  subsection.  Hence,  their  support 
for  such  training  in  general  should  be 
viewed  to  indicate  that  the  potential 
is  good  if  it  is  used  appropriately. 

The  geographic  data  base  used  during 
this  experiment  was  that  of  a  generic 
port,  Lotsedorf.  The  characteristics 
of  this  port  were  specifically  devel¬ 
oped  to  create  the  desired  training 
encounter  situations;  to  enable  the 
conduct  of  a  structured  training  pro¬ 
gram,  and  to  enable  the  collection  of 
desired  empirical  data.  It  should  be 
noted  that  this  port  was  not  developed 
to  be  fully  representative  of  the 
range  of  piloting  conditions  likely  to 
be  encountered  by  the  participating 
pilots  in  their  respective  pilot  re¬ 
gions,  and  certainly  is  not  represen¬ 
tative.  Nevertheless,  the  Lotsedorf 
port  was  believed  to  embody  sufficient 
characteristics  as  part  of  this 
experimental  training  program  to  be 
representative  of  the  generic  port 
concept  for  pilot  training.  The  simu¬ 
lator-based  training  of  pilots  in 
their  own  port  data  base  would  cer¬ 
tainly  have  value  beyond  that  of  a 
generic  data  base,  although  the  latter 
was  also  believed  to  have  significant 
value  for  training  pilots.  The  costs 
associated  with  developing  a  large 
number  of  specific  port  data  bases  to 
train  various  groups  of  pilots  would 
likely  be  prohibitive,  thus  making  the 
generic  port  concept  an  attractive 
alternative.  Furthermore,  for  the 
purposes  of  this  experiment  pilots 
from  a  variety  of  ports  were  brought 
together,  this  also  necessitated  a 
generic  port  data  base.  Most  of  the 
pilots  (11  out  of  12)  supported  the 
use  of  a  generic  data  base  for  the 
training  of  both  apprentice  and  ex¬ 
perienced  pilots.  It  should  be  noted, 
however,  that  several  pilots  were  cri¬ 
tical  of  the  training  program's  par¬ 
tial  focusing  on  issues  that  are  not 


of  current  relevance  in  their  respec¬ 
tive  port  areas.  For  example,  several 
pilots  expressed  disinterest  in  learn¬ 
ing  about  the  use  of  racons  since 
these  devices  are  not  currently  avail¬ 
able  in  their  port  area.  Operational 
training  programs,  using  a  generic 
port  data  base,  should  certainly  focus 
on  those  issues  of  local  interest  to 
the  pilots  being  trained.  The  opinion 
was  generally  expressed  that  the 
generic  port  concept  is  good  if  that 
generic  port  has  characteristics 
(e.g.,  bank  effects;  racons;  etc.) 
that  are  relevant  to  the  particular 
pilot's  port.  Hence,  valuable  train¬ 
ing  could  be  provided  in  the  generic 
port  context,  when  addressing  issues 
of  local  concern.  This  finding  sug¬ 
gests  that  a  single  we  11 -developed 
generic  port  data  base  may  be  adequate 
for  effective  training  use  by  most 
pilots,  substantially  reducing  the 
potential  cost  of  such  training. 

Specific  Training  Program  Content 

A  variety  of  comments  and  recommenda¬ 
tions  were  made  by  the  pilots  regard¬ 
ing  the  specific  content  of  the  expe¬ 
rimental  training  program  conducted  at 
CAORF.  The  detailed  response  data  are 
contained  in  Appendix  A.  Several 
specific  findings  are  discussed  below. 

Advanced  Instrumentation.  The  Ad¬ 
vanced  Instrumentation  module,  as 
noted  earlier,  was  generally  well 
received.  Several  pilots  indicated 
they  felt  it  had  the  greatest  poten¬ 
tial  for  simulator-based  training. 
Several  pilots  stated  that  the  length 
of  this  module,  as  conducted,  was  in¬ 
sufficient.  This  is  certainly  likely 
to  be  the  case,  since  the  module  was 
developed  to  address  a  relatively 
large  amount  of  material  in  a  brief 
period  of  time,  due  to  experimental 
considerations.  A  production  training 
module  that  would  address  advanced 
instrumentation  should  probably  span  a 
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considerably  longer  period  of  time. 
Pilot  comments  regarding  insufficient 
length  addressed  both  the  classroom 
and  the  simulator  sessions.  Addition¬ 
al  comments  about  this  module  were: 

•  Several  pilots  criticized  the 
module  as  not  addressing  issues 
that  are  directly  pertinent  to 
their  port  (e.g.,  racons  are  un¬ 
available  in  their  port). 

•  Improvements  to  the  course  recom¬ 
mended  by  the  pilots  included:  (1) 
bringing  the  ship  all  the  way  into 
the  port  at  once,  as  opposed  to 
doing  it  in  segments;  (2)  providing 
more  situation-type  of  experiences, 
as  opposed  to  the  more  sterile 
practice  exercises  that  were  ad¬ 
dressed;  (3)  providing  more  prac¬ 
tice  time;  (4)  providing  more  dif¬ 
ficult  scenarios;  and  (5)  providing 
more  port-specific  problems  (i.e., 
not  geography  specific,  but  condi¬ 
tions  more  specific  to  their  local 
ports). 

•  The  pilots  recommended  additional 

areas  of  Advanced  Instrumentation 
that  simulator-based  training  might 
address:  (1)  advanced  portable 

electronic  navigation  aids,  such  as 
the  Portable  LORAN  Assist  Device 
(PLAD);  (2)  advanced  radar  plotting 
aids  (ARPA);  (3)  bow  thruster  indi¬ 
cator;  (4)  controllable  pitch  pro¬ 
peller;  (5)  portable  fathometer; 
(6)  advanced  radar  techniques;  (7) 
radio  direction  finder;  and  (8)  use 
of  Advanced  Instrumentation  with 
bank  effects. 

Emergency  Shiphandling.  Most  pilots 
generally  felt  that  the  Emergency 
Shiphandling  module  was  worthwhile  as 
conducted  and  that  the  Dutch  turning 
around  technique  was  useful.  Several 
pilots,  however,  expressed  the  opinion 
that  the  particular  turning  around 


technique  addressed  was  either  not 
their  concept  of  Emergency  Ship¬ 
handling,  unrealistically  easy,  or  too 
clinical.  Several  pilots  felt  very 
strongly  about  the  value  of  simulator- 
based  Emergency  Shiphandling  training, 
both  in  the  form  provided  during  this 
program,  as  well  as  with  regard  to 
other  aspects  of  Emergency  Ship¬ 
handling  not  addressed  by  this  pro¬ 
gram.  Several  pilots  suggested  that 
this  is  the  major  area  of  potential 
benefit  for  the  simulator-based  train¬ 
ing  of  pilots.  Several  comments  re¬ 
garding  this  module  included: 

•  The  simulator  provides  a  safe  situ¬ 
ation  for  developing  skills  and 
trying  new  techniques. 

t  More  time  should  be  spent  on  the 
bridge,  rather  than  in  the  class¬ 
room;  the  converse  was  also  cited. 

•  Several  pilots  felt  the  program 
length  was  about  right,  while 
others  felt  it  should  be  expanded 
to  address  other  types  of  emergency 
situations. 

•  Several  pilots  felt  that  the  turn¬ 
ing  around  technique  took  too  much 
time;  they  would  like  to  see  more 
rapid  and  varied  situations. 

•  The  ship's  handling  characteris¬ 
tics,  as  modeled,  were  generally 
felt  to  be  too  good  for  developing 
the  emergency  shiphandling  skills 
addressed. 

•  Several  course  improvements  were 

recommended:  (1)  both  shorter  and 

longer  program  lengths;  (2)  a  wider 
variety  of  emergency  situations,  as 
well  as  exposure  to  a  larger  number 
of  situations;  (3)  shorter  duration 
situations;  and  (4)  a  range  of  as¬ 
sociated  conditions  under  which  to 


practice  the  maneuvering,  such  as 
poor  visibility,  a  more  narrow 
channel,  more  current,  and  time 
constraints. 

•  Additional  areas  of  Emergency  Ship¬ 
handling  training  that  were  recom¬ 
mended  included:  (1)  comparison  of 
average  and  poor  handling  ships  in 
similar  scenarios;  (2)  use  of  emer¬ 
gency  anchoring;  (3)  addressing 
rudder  failure  conditions;  (4)  ad¬ 
dressing  power  failure  conditions; 
(5)  creation  of  scenarios  that  re¬ 
quire  very  quick  thinking;  (6)  ad¬ 
dressing  crosscurrents  with  course 
changes;  (7)  addressing  winds;  (8) 
requiring  anchoring  in  a  short 
time;  (9)  addressing  speed  together 
with  anchoring;  (10)  addressing 
bank  effects;  (11)  addressing  ship/ 
ship  interaction;  (12)  turning 
around  in  an  anchorage;  and  (13) 
addressing  fog  onset  with  oncoming 
traffic. 

Decisionmaking.  The  pilots  had  re¬ 
latively  poor  experience  with  the  De¬ 
cisionmaking  module  due  to  the  simula¬ 
tor  problems.  Nevertheless,  a  number 
of  observations  were  made  by  eight  of 
the  pilots  that  had  some  experience 
with  this  module.  They  include  the 
following: 

•  Several  pilots  felt  this  module  was 
good,  since  it  addressed  specific 
situations  in  a  realistic  context. 

•  The  ownship  handling  characteris¬ 
tics  were  called  out  as  being  too 
good,  and  hence  somewhat  unrealis¬ 
tic. 

•  Several  pilots  felt  that  this 
module  should  be  combined  with  the 
emergency  shiphandling  module,  such 
that  the  emergency  training  would 
be  placed  in  the  context  of  the 
situations  that  occurred  in  this 
module. 


•  More  local  problems  were  requested 
by  several  pilots. 


•  The  course  improvements  recommended 
included:  (1)  having  only  partial 
astern  rpm  available;  (2)  use  of  a 
poor  handling  vessel  for  ownship; 
(3)  comparison  of  various  ship 
types  and  handling  characteristics; 
and  (4)  use  of  more  situations  that 
require  stopping. 


•  Additional  areas  of  Decisionmaking 
that  were  recommended  included: 
(1)  mechanical  breakdowns  (engine, 
steering);  (2)  techniques  for  stop¬ 
ping  the  ship  completely;  (3)  a 
situation  with  a  small  boat  running 
across  ownship' s  bow;  (4)  loose 
steering;  (5)  quartermaster  error; 
(6)  bank  effects;  (7)  abrupt  dives 
toward  the  bank,  shoal,  and  dock; 
and  (8)  having  a  meeting  ship  lose 
power  and/or  steering. 

The  above  observations  provide  a  num¬ 
ber  of  recommendations  relevant  to  the 
further  development  of  the  three 
training  modules  addressed  in  the  ex¬ 
perimental  training  program.  Commer¬ 
cial  training  establishments,  as  well 
as  pilot  groups,  would  benefit  from 
the  careful  consideration  of  these 
comments  with  regard  to  the  develop¬ 
ment  of  training  programs  for  pilots. 

Simulator-based  Training  Potential 

The  major  issues  of  concern  to  this 
investigation  have  been:  (1)  the  ef¬ 
fectiveness  of  simulator-based  train¬ 
ing  for  pilots;  (2)  the  pilot  skill 
areas  for  which  simulator-based  train¬ 
ing  would  be  effective;  and  (3)  the 
pilot  experience  levels  for  which 
simulator-based  training  would  be  ef¬ 
fective.  The  empirically  derived  re¬ 
sults  agree  well  with  the  question¬ 
naire  results  on  each  of  these  issues. 


The  pilots  participating  in  this 
experiment  generally  felt  that 
simulator-based  training  does  have 
potential  for  both  experienced  and 
apprentice  pilots. 

The  content  of  the  training  pre¬ 
sented  to  pilots  should  address 
issues  of  local  interest,  although 
not  necessarily  in  a  local  geo¬ 
graphical  -context;  that  is,  the 
particular  issues,  conditions, 
etc.,  should  be  of  specific  local 
interest,  although  the  port  could 
be  generic. 

Different  types  of  training  should 
be  provided  to  pilots  of  differing 
experience  levels  (e.g.,  appren¬ 
tices,  limited  experienced  pilots, 
extensively  experienced  pilots). 

The  ship  model  characteristics 
(ship  hydrodynamics,  environmental 
hydrodynamics,  etc.)  are  particu¬ 
larly  critical,  and  require  exten¬ 
sive  validation/modif ication  by 
pilots  prior  to  training.  Pilots 
are  especially  sensitive  to  the 
vessel  model's  handling  character¬ 
istics.  Furthermore,  the  pilots 
generally  expect  a  poor  handling 
ship  in  a  simulator-based  training 
program,  since  it  represents  the 
extreme  set  of  conditions  which 
they  believe  are  more  appropriate 
to  such  training. 

Pilots  expressed  an  interest  in 
having  a  large  number  of  short 
runs,  rather  than  several  relative¬ 
ly  long  runs.  Although  this  may  be 
at  variance  with  their  actual  ex¬ 
periences  at-sea,  it  provides  the 
pilots  with  an  opportunity  to  ex¬ 
plore  a  wide  range  of  conditions 
and  possible  responses. 

The  length  of  the  training  modules 
were  generally  considered  to  be  too 
short.  This  was  expected,  as  a 


result  of  the  cost/effective  nature 
of  constructing  an  experiment.  The 
primary  intention  of  this  program 
was  to  generate  as  much  useful  in¬ 
formation  as  possible  from  the  ex¬ 
periment,  necessitating  allowance 
of  only  the  minimum  amount  of  time 
for  the  various  piloting  issues 
addressed  during  training.  A  com¬ 
mercial  training  program  should 
spend  considerably  more  time  ad¬ 
dressing  the  issues  of  each  module. 

•  The  pilots  also  indicated  that  the 
use  of  simulator -based  training 
should  be  to  augment  their  current 
training  programs,  and  not  simply 
be  used  in  place  of  certain  aspects 
of  their  current  training.  This 
statement  should  not  be  interpreted 
that  present  training  programs  are 
deficient;  rather  it  indicates  the 
healthy  attitude  of  professionals 
seeking  to  improve  the  quality  of 
their  service. 

•  The  generic  data  base  was  found  to 
be  generally  well -accepted  by  the 
pilots,  with  specific  comments  cit¬ 
ing  the  desire  for  port-specific 
training  within  a  generic  port  data 
base  if  conducted  in  an  appropriate 
manner. 

In  summary,  the  questionnaire  opinions 
correspond  very  well  with  the  experi¬ 
mental  results.  The  empirical  data 
unequivocally  demonstrate  that  simula¬ 
tor-based  training  can  be  effective 
for  pilots.  Pilot  observations  con¬ 
firm  this  finding,  and  support  the 
potential  effectiveness  of  such  train¬ 
ing.  Pilot  support  of  simulator-based 
training,  based  on  the  small  sample 
size  used  during  this  investigation, 
was  not  unequivocal.  Rather,  it  was 
highly  qualified.  Simulator-based 
training  is  supported  as  having  poten¬ 
tial  by  pilots,  but  only  when  properly 
used  in  direct  correspondence  to  local 
conditions  and  issues,  and  with  regard 


CHAPTER  5 


CONCLUSIONS 


The  major  issues  of  interest  to  this 
investigation  were:  (1)  the  potential 
effectiveness  of  simulator-based 
training  for  pilots;  (2)  the  pilot 
skill  areas  for  which  simulator-based 
training  is  potentially  effective;  and 
(3)  the  pilot  experience  levels  for 
which  simulator-based  training  would 
be  effective.  In  addition  to  address¬ 
ing  these  issues,  this  experiment  has 
addressed  additional  secondary  is¬ 
sues.  Conclusions  regarding  each  of 
these,  as  developed  in  the  previous 
Results  and  Discussions  sections,  are 
provided  below. 


1.  The  findings  of  this  experiment 
strongly  demonstrate  the  potential 
effectiveness  of  simulator-based 
training  for  pilots.  The  empirical 
data  showed  substantial  training 
gains  for  the  pilots.  This  was 
supported  by  the  observation  of  the 
pilots  themselves,  in  which  a  high 
proportion  felt  that  they  had 
gained  from  this  training  program, 
and  also  felt  that  simulator-based 
training  has  potential  for  improv¬ 
ing  pilot  skills.  Both  the  empiri¬ 
cal  data  and  the  pilot  opinions  are 
not  unequivocal  in  this  regard. 
They  show  strong  support  for  simu¬ 
lator-based  training,  but  do  not 
indicate  that  all  such  training 
would  be  either  desirable  or  effec¬ 
tive.  Rather,  these  findings  indi¬ 
cate  the  potential  for  such  train¬ 
ing  if  conducted  under  the  right 
conditions  and  conducted  appropri¬ 
ately. 


2.  Simulator-based  training  is  effec¬ 
tive  for  addressing  Advanced  In¬ 
strumentation  and  Emergency  Ship¬ 
handling  piloting  issues.  This 
conclusion  is  based  on  the  empiri¬ 
cal  data  which  showed  both  training 
modules  to  be  effective,  and  on  the 
opinions  of  the  pilots  which  indi¬ 
cated  the  value  and  potential  of 
these  training  modules. 

3.  Simulator-based  training  is  ex¬ 
pected  to  be  effective  for  most  of 
the  additional  training  areas  indi¬ 
cated  in  this  investigation  (see 
paragraph  2.1,  Project  Approach). 
This  conclusion  is  reached  on  the 
basis  of  the  correspondence  between 
(1)  the  findings  of  the  structured 
training  analysis  that  identified 
the  areas  of  piloting  skill  that 
appear  to  have  potential  for  simu¬ 
lator-based  training,  (2)  the 
strong  training  effectiveness  find¬ 
ings  for  the  two  selected  modules, 
(3)  the  opinions  of  the  pilots  sup¬ 
porting  the  two  selected  modules, 
and  (4)  the  pilots'  opinions  re¬ 
garding  the  potential  of  the  third 
module  for  training  apprentice  and 
limited  experienced  pilots.  The 
strong  agreement  regarding  these 
issues  supports  the  contention  that 
similar  results  would  likely  per¬ 
tain  to  the  other  potential  areas 
for  simulator-based  training  iden¬ 
tified  in  paragraph  2.1. 

4.  Simulator-based  training  appears  to 
have  potential  for  apprentice 
pilots,  licensed  pilots  with  li¬ 
mited  experience,  and  licensed 


pilots  with  extensive  experience. 
The  particular  aspects  of  piloting 
skills  for  which  such  training  will 
be  effective  is  likely  to  differ 
substantially  for  each  of  these 
groups.  Both  the  empirical  data 
and  the  opinions  of  the  pilots  in¬ 
dicate  differential  training  effec¬ 
tiveness  for  different  groups  of 
pilots  ( i . e . ,  extensive  experience 
versus  limited  experience).  Unfor¬ 
tunately,  the  empirical  data  are 
insufficient  to  identify  the  most 
appropriate -  areas  of  training  for 
the  different  levels  of  pilot  ex¬ 
perience.  Nevertheless,  they  do 
support  the  contention  that  the 
benefits  to  be  gained  from  simula¬ 
tor-based  training  will  vary  for 
each  group  of  pilots  with  regard  to 
the  particular  shiphandling  skills 
addressed. 

5.  The  effectiveness  of  simulator- 
based  training  is  likely  to  be 
heavily  dependent  on  the  particular 
piloting  issues  associated  with 
each  port,  and  on  the  specific 
backgrounds  of  the  individual 
pilots.  This  conclusion  has  been 
reached  on  the  basis  of  the  empiri¬ 
cal  data  showing  differences  be¬ 
tween  the  groups  of  pilots,  obser¬ 
vations  of  substantial  individual 
differences  within  groups  during 
the  experiment,  and  the  opinions  of 
the  pilots.  For  this  reason,  simu¬ 
lator-based  training  cannot  be  said 
to  be  effective  for  all  pilots  in 
all  ports.  Rather,  the  effective¬ 
ness  of  such  training  will  depend 
on  the  skill  areas  being  addressed, 
particular  port  characteristics, 
and  the  individual  pilot/trainee 
background. 

6.  Pilot  opinion  of  simulator-based 
training  is,  for  the  most  part, 
positive.  Although  pilot  opinion 
has  been  generally  thought  to  be 
negative  toward  simulator-based 


training,  this  was  not  found  to  be 
the  case.  Rather,  the  apparently 
negative  opinion  stems  from  (1)  the 
belief  that  pilot  training  needs 
are  beyond  simulator  capabilities 
(i.e.,  which  they  are  for  certain 
skills,  although  other  skills  can 
be  properly  addressed  via  simula¬ 
tor-based  training);  (2)  the  belief 
that  simulator-based  training 
should  not  be  used  in  place  of  cur¬ 
rent  training  practices  (rather,  it 
has  potential  value  to  augment  cur¬ 
rent  training  programs  and  hence 
upgrade  piloting  skills);  and  (3) 
the  belief  that  the  application  of 
simulator-based  training  will  be 
required  of  pilots  and  misdirected 
as  a  result  of  regulators  not  being 
sensitive  to  particular  pilot  local 
needs  and  practices.  When  address¬ 
ing  the  potential  merits  of  simula¬ 
tor-based  training  unencumbered  by 
these  other  considerations  the 
pilots  were  generally  very  positive. 

7.  A  generic  data  base  for  pilot 
training  can  be  effective.  The 
empirical  data  demonstrated  the 
effectiveness  of  pilot  training 
using  a  generic  data  base.  Pilot 
opinion  was  largely  in  agreement 
with  the  effectiveness  of  a  generic 
data  base.  However,  the  particular 
situations,  skills,  conditions, 
etc.,  addressed  during  the  training 
program  should  be  tailored  to  the 
pilots'  local  port  characteris¬ 
tics.  In  other  words,  the  geogra¬ 
phy  need  not  be  the  same,  but  the 
issues  addressed  (e.g. ,  bank  ef¬ 
fects)  should  be  those  of  particu¬ 
lar  relevance  in  the  pilots'  port. 
This  finding  is  particularly  impor¬ 
tant,  since  it  strongly  suggests 
that  effective  pilot  training  can 
be  achieved  without  the  substantial 
expense  of  modeling  each  pilots' 
port.  It  should  also  be  noted, 
that  this  finding  does  not  mean  to 
imply  that  all  piloting  skills  ad- 


dressable  via  simulator-based 
training  can  be  effectively 
achieved  using  a  generic  port. 
Rather,  the  generic  port  may  be 
appropriate  for  training  a  large 
proportion  of  such  skills. 

8.  The  training  system  characteristics 
are  particularly  important  for  the 
training  of  pilots.  This  impor¬ 
tance  appears  greater  than  that  for 
the  training  of  deck  officers.  The 
training  program  content  should  be 
specifically  tailored  to  meet  the 
needs  of  each  group  of  pilots. 
This  would  include  the  skills  being 
addressed,  as  well  as  the  situa¬ 
tions  and  conditions  contained  in 
the  exercises.  A  relatively  large 
number  of  short-duration  exercises 
should  be  employed  in  the  training 
program.  These  should  focus  on  the 
extreme  situations  likely  to  be 
encountered  by  the  pilots.  They 
should  provide  the  pilots  with  a 
relatively  difficult  challenge,  and 
enable  them  to  practice  a  variety 
of  techniques  that  safety  and  other 
considerations  would  not  allow  try¬ 
ing  at-sea.  Finally,  the  level  of 
simulation  fidelity  is  very  criti¬ 
cal.  The  pilots  were  unanimously 
critical  of  the  ship  hydrodynamics, 
even  though  considerable  care  and 
development  had  been  focused  on 
providing  an  appropriate  ship 
model.  Similar  concern  had  been 
experienced  at  other  facilities 
training  pilots  (e.g..  Europort 
pilots  trained  at  Wageningen).  The 
pilots  are  extremely  sensitive  to 
the  hydrodynamic  characteristics  of 
the  vessel,  and  would  prefer  a  re¬ 
latively  poorly  handling  vessel 
that  represents  the  more  difficult 
end  of  the  scale.  Although  similar 
criticism  was  not  placed  on  the 
environmental  aspects  of  the  simu¬ 
lation,  it  is  expected  that  this 


conclusion  would  apply  equally  to 
other  aspects  of  simulation  fide¬ 
lity  closely  aligned  with  pilot 
tasks  (e.g.,  bank  interaction  with 
ownship) . 

9.  The  mechanics  of  the  training  pro¬ 
gram,  in  terms  of  class  size 
(i.e.,  four  pilots),  mixture  of 
extensive  and  limited  experienced 
pilots  (i.e.,  usually  two  in  each 
group),  the  case  study/seminar 
training  method  used  for  programs 
involving  experienced  students, 
the  coordinator  roles  provided  by 
the  pilot  consultants  and  so  on, 
represent  an  effective  training 
program  strategy  for  the  conduct 
of  piloting  training.  This  con¬ 
clusion  is  based  on  the  careful 
analysis  arid  development  work 
leading  to  the  training  program, 
the  strong  empirical  results  de¬ 
monstrating  the  effectiveness  of 
the  training  program,  and  the 
opinions  of  the  pilots  showing 
support  for  the  training  program. 

10.  A  governmental  requirement  for  all 
pilots  to  participate  in  simula¬ 
tor-based  training  at  this  time 
would  likely  be  ineffective.  This 
conclusion  is  reached  on  the  basis 
of  the  opinions  of  the  pilots. 
Although  they  were  generally 
strongly  supportive  of  the  poten¬ 
tial  for  simulator-based  training 
as  a  means  of  improving  piloting 
skills,  they  were  equally  strongly 
adverse  to  the  concept  of  such 
training  being  required.  The 
findings  of  this  investigation, 
although  strongly  supportive  of 
the  potential  effectiveness  of 
simulator-based  training  for 
pilots,  also  suggest  that  the 
effectiveness  can  vary 

substantially  depending  on  many 
factors. 
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CHAPTER  6 


RECOMMENDATIONS 


As  a  result  of  the  conduct  and  analy¬ 
sis  of  the  experimental  training  pro¬ 
gram  described  within  this  report, 
several  pertinent  recommendations  have 
been  identified  and  discussed  below. 
It  should  be  noted  that  these  recom¬ 
mendations  are  the  authors'  recommen¬ 
dations  relating  to  the  implementation 
and  future  research  of  marine  pilot 
simulator-based  training.  They  do  not 
necessarily  reflect  the  views  or  in¬ 
tentions  of  either  the  U.S.  Coast 
Guard  or  the  Maritime  Administration 
who  sponsored  this  research. 


1.  Government  agencies  should  posi¬ 
tively  encourage  pilots  to  avail 
themselves  of  simulator-based 
training.  Such  training  should  not 
be  required.  As  evidenced  by  this 
experiment,  simulator-based  train¬ 
ing  of  pilots  is  in  the  develop¬ 
mental  stage.  It  has  been  examined 
here,  and  is  believed  to  have  po¬ 
tential  for  the  acquisition,  main¬ 
tenance,  an  improvement  of  selected 
piloting  skills.  It  should,  there¬ 
fore,  be  considered  as  a  means  for 
augmenting/upgrading  the  current 
training  practices  of  pilot  organi¬ 
zations.  It  is  further  recommended 
that  the  appropriate  use  of  simula¬ 
tor-based  training  be  left  to  the 
discretion  of  each  pilot  organiza¬ 
tion.  This  would  enable  each  or¬ 
ganization  to  cost  effectively  use 
simulator-based  training  as  one  me¬ 
dium  of  a  multi-media  approach  to 
meet  their  particular  training 
needs. 


2.  An  investigation  should  be  under¬ 
taken  to  identify  those  pertinent 
areas  of  shiphandling  skill  for 
which  cost/effective  simulator- 
based  training  can  be  provided  for 
pilots.  The  investigation  should 
thoroughly  document  those  skill 
areas  having  potential  benefit  from 
simulator-based  training,  and 
determine  those  skill  areas  for 
which  simulator-based  training 
would  be  effective.  The  final 
evaluation  of  effectiveness  should 
be  based  on  a  combination  of  struc¬ 
tured  subjective  analysis  and  em¬ 
pirical  research.  The  intent  would 
be  to  develop  a  thoroughly  docu¬ 
mented  catalog  of  pilot  skill  areas 
amenable  to  simulator -based  train¬ 
ing.  The  first  step  of  this  cata¬ 
log  is  the  guidelines  document  de¬ 
veloped  by  this  project  addressing 
simulator-based  training  system 
design  for  pilots.  The  experiment 
reported  herein  conducted  a  limited 
analysis  of  this  type,  identifying 
several  possible  areas  of  piloting 
skill  amenable  to  simulator-based 
training.  A  much  more  thorough 
investigation  directly  designed  for 
this  purpose  is  recommended  to  in¬ 
vestigate  and  identify  those  areas 
for  which  simulator-based  training 
would  be  cost/effective  for  pilots. 

3.  Several  pilot  groups  should  be 
identified  for  which  simulator- 
based  training  would  be  particular¬ 
ly  effective,  and  also  acceptable 
to  the  pilots.  Appropriate  train¬ 
ing  programs  should  be  set-up  on  a 


cost-shared  basis  between  the 
pilots  and  the  government.  The 
purpose  of  these  initial  training 
programs  would  be  twofold:  (1)  to 
achieve  effective  pilot  training 
where  needed;  and  (2)  to  encourage 
the  use  of  simulator-based  training 
on  the  part  of  pilots  by  providing 
prototype  training  programs  in  cer¬ 
tain  areas.  This  will  help  to  edu¬ 
cate  pilots  regarding  the  cost/ 
effectiveness  of  simulator-based 
training,  as  well  as  its  accept¬ 
ability.  A  major  problem  today 
with  the  acceptance  of  simulator- 
based  training  is  the  lack  of  dir¬ 
ect  knowledge  by  pilots  of  the 
capabilities  and  limitations  of 
this  training  medium. 

4.  A  high  quality  generic  port  data 
base  should  be  developed  that  will 
contain  representative  critical 
characteristics  of  ports  around  the 
country  as  they  pertain  to  the 
simulator-based  training  of  speci¬ 
fic  pilot  skill  areas.  This  gener¬ 
ic  data  base  should  have  the  char¬ 
acteristics  to  become  a  standard¬ 
ized  training  port  for  pilots. 
The  data  base  should  be  developed 
so  as  to  enable  commercial  training 
facilities  to  transpose  its  charac¬ 
teristics  into  an  appropriate  simu¬ 
lation  data  base  (i.e.,  visual  and 
radar  images,  hydrodynamics).  A 
catalog  of  the  generic  port  charac¬ 
teristics  should  be  crossed-indexed 
with  the  appropriate  ports  around 
the  country  to  provide  guidance 
information  concerning  potential 
use.  This  development  should  in¬ 
clude  the  validation  of  the  generic 
port  characteristics. 

5.  Government  agencies  should  provide 
assistance  to  pilot  organizations 
to  help  them  in  the  development  and 
use  of  simulator-based  training 
programs.  In  addition  to  the  fi¬ 
nancial  assistance  noted  above,  it 


is  recommended  that  the  government 
provide  technical  training  assis¬ 
tance  for  the  development  of  effec¬ 
tive  training  programs. 

6.  An  effort  should  be  conducted  to 
thoroughly  develop  several  repre¬ 
sentative  ship  hydrodynamic 
models.  This  developmental  effort 
should  include  at-sea  validation  as 
necessary.  Its  purpose  should  be 
to  develop  several  representative 
models  that  have  met  the  approval 
of  pilot  organizations,  and  which 
can  be  made  available  to  commercial 
training  establishments.  The 
hydrodynamic  fidelity  of  the  vessel 
and  the  environment  are  of  major 
concern  to  pilots,  as  evidenced 
during  this  investigation.  The 
lack  of  acceptable  ship  models 
would  likely  represent  a  major 
stumbling  block  to  the  effective 
use  of  simulator-based  training  by 
pilots.  Hence,  it  is  in  the  inter¬ 
est  of  the  industry  and  maritime 
safety  that  models  be  developed 
which  meet  the  approval  of  pilot 
organizations,  and  hence  assist  in 
facilitating  the  simulator-based 
training  of  pilots. 

7.  Organizations  that  intend  to  use 
all  or  parts  of  the  experimental 
pilot  training  program  developed 
for  this  investigation  should  use 
the  data  reported  herein,  and 
otherwise  available,  to  update 
those  training  modules.  Effective 
use  of  this  material  would  require 
modification  of  the  modules  in  ac¬ 
cordance  with  the  observations  of 
the  pilots  and  the  project  staff. 
This  is  considered  essential  for 
effective  use  of  this  material. 

8.  At-sea  validation  of  the  findings 
achieved  during  the  Training  and 
Licensing  Project  should  be  under¬ 
taken  to  provide  the  bottom-line 


information  regarding  the  effec¬ 
tiveness  of  simulator-based  train¬ 
ing.  The  findings  from  this  expe¬ 
riment  and  the  earlier  investiga¬ 
tions  during  this  project  have  do¬ 
cumented  benefits  and  limitations 
of  simulator-based  training  of 
mariners.  These  findings,  however, 
have  been  confined  to  the  simulator 
environment.  As  a  result,  the  use 
of  simulator-based  training  has 
been  growing  slowly,  even  though 


the  data  suggest  it  is  a  potential¬ 
ly  strong  medium  for  improving 
shiphandling  proficiency.  Demon¬ 
stration  of  the  effectiveness  of 
this  medium  for  improving  at-sea 
shiphandling  proficiency  is  a  ne¬ 
cessary  pre-requisite  for  achieving 
substantially  increased  industry 
use  of  this  powerful  tool  and  the 
resulting  benefits  to  shiphandling 
safety. 


APPENDIX  A 


DEBRIEFING  QUESTIONNAIRE 


The  following  questionnaire  will  be  administered  by  one  of  the  research  staff. 
Although  responses  to  many  questions  are  specified,  there  is  a  comment  space 
available  after  each  response.  The  staff  member  should  encourage  comments  by  the 
trainees. 

Instructions  to  Trainee: 

In  answering  the  following  questions,  please  do  not  hesitate  to  criticize  any 
aspect  of  the  training  program  here  at  CAORF  -  simulator,  seminar  coordinator, 
conditions,  etc.  Your  honest  answers,  both  negative  and  positive  will  be  of  help 
to  both  CAORF  and  this  project.  Be  specific  as  possible  when  asked  for  comments. 

MODULE  I:  ADVANCED  INSTRUMENTATION 

1.  On  a  scale  of  one  to  ten,  ten  being  excellent,  how  would  you  rate  the  "advanced 
instrumentation"  segment  of  the  training  program? 

123456789  10 

Comments:  7.58  Average _ 


2.  Was  the  amount  of  time  allocated  to  the  use  of  racons  appropriate? 

YES  NO 


Comments :  8/12  Yes 


3.  Was  the  amount  of  time  allocated  to  the  use  of  the  rate-of-turn  indicator 
appropriate? 

YES  NO 

Comments:  10/12  Yes _ 


4.  Was  the  time  allocated  to  the  use  of  the  doppler  speed  log  appropriate? 

YES  NO 

Comments:  11/12  Yes 


5.  Do  you  feel  that  you  learned  something  valuable  which  you  can  use  in  your 
pilotage  port  from  the  "advanced  instrumentation"  segment  of  the  training 
program? 

YES  NO 

Comments :  11/12  Yes _ 


6.  What  other  types  of  "advanced  instrumentation"  should  be  considered  for  this 
type  of  training? 

(Various) 
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7.  Do  you  feel  that  such  training  in  "advanced  instrumentation"  would  be  valuable 
for  apprentice  pilots? 

YES  NO 


8.  Do  you  feel  that  periodic  optional  training  in  "advanced  instrumentation”  would 
be  valuable  for  experienced  pilots  to  keep  abreast  of  the  recent  developments 
in  such  bridge  equipment? 


YES  NO 


Comments:  11/12  Yes 


9.  If  an  "advanced  instrumentation"  module  was  offered  at  a  commercial  simulator 
facility,  what  changes  should  be  implemented? 

* 


10.  Was  the  simulation  of  the  handling  characteristics  of  the  80,000  dwt  tanker 
suitable  for  the  "advanced  instrumentation"  training? 

YES  NO 


Comments:  9/12  Yes 


MODULE  II: 

EMERGENCY  SHIPHANDLING 

11.  On  a  scale  of  one  to  ten,  ten  being  excellent,  how  would  you  rate  the 
"emergency  shiphandling"  segment  of  the  training  program? 

1  2  3  4  5 

6 

7  8  9 

10 

Comments :_ 

6.58  Average 

12.  Do  you  concur  that  such  training  in  turning  around  within  confined  channels  is 
a  valuable  training  exercise  for  developing  selected  shiphandling  skills  that 
may  be  appropriate  in  emergency  situations? 


Comments: _ 11/12  Yes 


13.  What  other  types  of  exercises  do  you  believe  could  be  appropriate  for 
developing  selected  shiphandling  skills  that  may  be  appropriate  in  emergency 
situations? 


Comments:  (Various] 


14.  Do  you  feel  that  such  training  in  "emergency  shiphandling"  skills  would  be 
valuable  for  apprentice  pilots? 


Comments:  11/12  Yes 


15.  Do  you  feel  that  periodic  optional  training  in  "emergency  shiphandling"  would 
be  valuable  for  experienced  pilots  in  order  to  refresh  seldom-used  skills? 


Comments :  9/12  Yes 


YES  NO 


16.  If  an  "emergency  shiphandling"  module  was  offered  at  a  commercial  simulator 
facility,  what  changes  should  be  implemented? 

Comments :  (Various) _ 


17.  Was  the  simulation  of  the  handling  characteristics  of  the  80,000  dwt  tanker 
suitable  for  the  "emergency  shiphandling"  training? 

YES  NO 

Comments :  8/12  Yes  _ _ _ 


18.  Was  there  adequate  time  allocated  within  the  training  program  for  refining 
your  skill  in  turning  around  the  80,000  dwt  tanker  within  the  confined  channel? 

YES  NO 


Comments :  12/12  Yes 


MODULE  III:  DECISIONMAKING 


19.  On  a  scale  of  one  to  ten,  ten  being  excellent,  how  would  you  rate  the 
“decisionmaking"  segment  of  the  training  program? 

123456789  10 

Comments :  6.29  Average _ 


I 

I 


20.  Do  you  feel  that  you  learned  something  valuable  which  you  can  use  in  your 
pilotage  port  from  the  "decisionmaking"  segment  of  the  training  program? 

YES  NO 

Comments :  3/8  Yes _ 


21.  What  other  types  of  decisionmaking  situations  do  you  believe  would  be 
appropriate  for  development/refinement  via  such  simulator  training? 

Comments :  (Various) _ 


22.  no  you  feel  that  such  "decisionmaking"  training  would  be  valuable  for 
apprentice  pilots? 


YES  NO 


Comments :  7/8  Yes 


23.  Do  you  feel  that  such  "decisionmaking"  training  would  be  valuable  for  licensed 
pilots  with  limited  experience? 

YES  NO 


Comments :  8/8  Yes 


24.  Do  you  feel  that  such  "decisionmaking"  training  would  be  valuable  as  a 
refresher  for  licensed  pilots  with  extensive  experience? 

YES  NO 


Comments :  3/8  Yes 


25.  If  a  "decisionmaking"  module  was  offered  at  a  commercial  simulator  facility, 
what  changes  should  be  implemented? 

Comments :  (Various) _ 


26.  Was  the  simulation  of  the  handling  characteristics  of  the  80,000  dwt  tanker 
suitable  for  the  "decisionmaking"  training? 

YES  NO 


Comments :  4/8  Yes 


GENERAL 

27.  Do  you  believe  that  shiphandling  simulators  have  a  role  in  the  training 
process  of  apprentice  pilots? 

YES  NO 

Corrments :  12/12  Yes _ 


28.  Do  you  believe  that  shiphandling  simulators  have  a  role  in  the  training 
process  of  licensed  pilots? 


YES  NO 


29.  Would  you  recommend  that  another  member  of  your  pilot  association  attend  a 
similar  simulator-based  training  program  at  a  commercial  training  facility  if 
the  tuition  cost  was  $3000? 


YES  NO 


Comments :  4/12  Yes 


30.  Would  you  recommend  that  another  member  of  your  pilot  association  attend  a 
similar  simulator-based  training  program  at  a  commercial  training  facility  if 
the  tuition  cost  was  $1000? 

YES  NO 


Comments :  7/12  Yes 


31.  What  is  the  maximum  duration  that  a  simulator  training  program  could  have 
without  seriously  impacting  your  job  and  personal  schedule? 

2  Days  1  Week  2  Weeks  Other: _ 

Comments :  6—2  Weeks;  4  —  1  Week;  1  --  2  Days 


32.  Do  you  now  believe  that  marine  pilots  can  receive  beneficial  training  on  a 
shiphandling  simulator  within  a  generic  port? 

YES  NO 


Comments :  11/12  Yes 


WY  v 
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